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Objectives of the study (summarized in Chapter I) were (1) to propose a 
task-oriented viewpoint in which the scientist actively participates as an integral part 
of the system in performing research tasks. (2) to identify major system planning 
decisions and criterion concepts for use in evaluation of proposed information 
systems, and (3) to construct a planning guide. Chapter II deals with functional 
requirements of a system. The task-oriented philosophy is described using a 
orant-supported research project as the task to be accomplished by the system. 
Chapter III is concerned with the technical description of a system. A system planning 
model was constructed. Chapter IV defines and classifies administrative functions and 
identifies management tasks. Chapter V describes the system planning and 
development process. This chapter identifies activities occurring and level of system 
description realizable during each of five planning and development phases. Appendix 
A gives a resea.*"ch project model. Appendix B includes a summary of functional, 
technical and administrative aspects of the observed information systems and a 
discussion of major characteristics associated with different information system 
models. (CC) 
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CHAPTER I 
INTRODUCTION 












I . Introduction 



A. Oblectlve and Scope 

One of the missions of the National Science Foundation (NSF) 
is to support the effective dissemination of scientific information (102). 
The Office of Science Information Service (OSIS) within NSF has been 
assigned a major responsibility for carrying out this mission. One 
part of this responsibility involves the awarding of information system 
development grants to universities and professional societies . The 
traditional role of OSIS with respect to these system development 
grants has been supportive and administrative in character. That is, 
financial support is provided to universities and professional societies 
through the administrative mechanisms of a grant. Within fairly broad 
policy guidelines and limited constraints provided by NSF, the grantee 
assumes primary responsibility for planning, executing, and documenting 
the grant-supported project. This traditional relationship between NSF 
and the universities/societies has proved to be a fairly harmonious 
and satisfactory one. However, an accelerating trend has been evolving 
most noticeably during the past two years which is necessitating an 
examination of this traditional relationship. This trend is briefly 
described below and discussed in greater detail in Section B of this 
chapter . 

The trend involves the broadening requirement of all govern- 
ment funded agencies and organizations to spell out in some detail 
the expected benefits , costs , research and development (R&D) time 
frame, and risks associated with system development projects submitted 
for fiscal support. There are at least two obvious factors contributing 
to this trend: (1) an increasing number of agencies and programs are 
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competing for the federal dollar and (2) a growing realization that 
the planning and development of complex systems, for example, 
computer-based information systems, is quite expensive, usually 
requires a number of years for the systems to be made operational , 
and generally entails a significant degree of uncertainty or risk 
regarding the eventual effectiveness of the developed system. 

Not only are there increasing numbers of organizations and 
programs requesting federal funding but also the systems being funded 
are becoming more complex and expensive in time, talent, and money 
to plan and develop. Given this situation, the decision problem of 
effectively allocating a finite amount of federal funds requires , during 
the entire system planning and development cycle, information and 
data on the projected benefits, costs, and risks as inputs to the 
decision process. The end product or objective of this decision 
process is to allocate funds to those information system alternatives 
which show the greatest likelihood of providing significant benefits within 
established cost constraints . The degree and breadth of prior planning 
needed to provide these types of inputs to the decision process has not 
been a normal procedure required in the past of those seeking federal 
grants . 

It is one thing to formulate a requirement within the domain 
of computer-based information systems . It is quite a different matter 
to effectively implement it. The accumulation to date of system 
planning and development experience has rather consistently revealed 
the existence of a gap between required decision inputs (relevant to 
projected costs, benefits, and risks) and actual performance in 
generating either comprehensive or, in some cases, meaningful 
decision inputs . 

Excluding the evident difficulty of forecasting the future as 
one must when planning new systems , there are at least two perhaps 
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not so obvious factors contributing to the system planning and develop- 
ment problem. 

First, the evaluation of alternative information system concepts 
or operational systems is made difficult by considering the system user 
as a recipient of the services of the system. This position forces the j 

system planner to view the user as the ultimate criterion source regarding 
the utility of the system. Given this orientation, the major alternative 
choices available to an investigator are either (1) verbal responses of the I 

user concerning the acceptability of the service and the relevance of the i 

retrieved information or (2) externally observable/measurable behavior of I 

the user which might be grouped under the general rubric "system usage" . I 

Both of the available choices have led, methodologically speaking, thorny 
existences . Recently published studies are described later in this chapter 
to highlight methodological difficulties encountered with this orientation. 
Furthermore, to effectively implement either choice, it is necessary for 
the system in question to be either operationally simulated or on-line in 
an operational status. However, the immediate problem facing the system 
planner is to obtain useful inputs when the system is in the earliest planning 
stages . During the very early planning stages , neither of the above choices 
are feasible; consequently, the system planner must either (1) design the 
system on the basis of verbal expressions of information needs supplied by 
members of the potential user population or (2) shift to a different orientation 
such as designing the system based or the characteristics of the information 
being published within a particular discipline or problem area . In this latter 
instance, the system planner has shifted from a user-need orientation to i 

an information- supply orientation. 

Second, the sheer complexity associated with planning and develop- j 
ing computer-based systems produces its own problems . These include the 
large number of system alternatives that are available, particularly at the 
component and subsystem levels; the potentially significant and multiple 








. The information systems of concern are restricted to complex 
computer-based systems designed to perform or contribute to the 
performance of a number of different functions j for example/ computational/ 
research management/ bibliographic/ clerical/ simulation. These complex 
information systems / even under ideal conditions / require a three to five 
year planning and development cycle. Finally/ the hardware and software 
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technology considered within the scope of this planning guide is well 
within the state-of-the-art with most of the components being "off- 
the-shelf" variety. 

, The "user", or more accurately, the participant population are 
scientists working in academic environments who hold positions at the 
assistant/associate professor level and who also have had two or three 
years research experience in a specific content area. Furthermore, 
the scientists are generally independent investigators with limited 
time to conduct research and limited research facilities at their 
disposal. Finally, their research work is supported by grants for 
the most part, and their research findings are generally reported at 
professional meetings and published in professional journals. 

. The scientist requirements of the computer-based information 
system are those involving the accomplishment of grant-supported research 
projects . The research project activities of interest range from the initial 
preparation of a proposal to the final submission of an approved article 
for publication. All tasks involved in the planning, conduct, and 
documentation of a grant- supported research project are to be considered 
as potential candidates for accomplishment by the integrated scientist/ 
computer-based system. 

. The computer-based system of concern to this planning guide 
is one which is either located on the university campus or accessible 
through remote input/output devices. In either case, the system- 
provided services are administratively supported by the university. 

B . Discussion of the Problem 

In Section A, an observation was made concerning the broadening 
requirement for government funded agencies and organizations to project 
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expected benefits, risks, and costs in their proposed R&D programs. 
Increasing competition for the federal dollar and steadily growing costs 
of conducting R&D projects were identified as two of the factors contribut- 
ing to this trend. In line with this observation, an article by Frederick 
Seitz in the September 1966 American Scientist Journal (84) discussed 
the need for an increasing rate of federal support of fundamental research 
to meet the rapidly rising costs. He also pointed out that the increasing 
needs of our society for money represents one of the reasons why the 
support of basic research operates, for the most part, on a fixed percent- 
age of the national income rather than on the level of research productivity 
that can be attained. At a quite recent meeting sponsored by the New York 
Academy of Sciences entitled "The Crisis Facing American Science" , 

Linus Pauling (19) contended that the reported 15% annual increase in 
federal support produced less than 7.5% increase in scientific activity 
because (1) research is being carried out on harder problems involving 
more complex and expensive equipment and facilities, (2) inflation 
reduces the purchasing power of each research dollar, and (3) wages 
and costs of services which support research programs have generally 
increased. 

At the same meeting, Walter S. Baer, fr n the Office of 
Science and Technology, also made a number of points relevant to 
this trend. First, federal grants for basic research are spent in 
many different ways — for graduate and post-doctoral assistantships, 
for faculty summer salaries, for equipment, for research materials, 
and for general overhead expenses . Some of these items are more 
critical and timely than others . Given the fact that federal support 
of R&D programs is levelling off, it is important that members of the 
scientific community help establish priorities to assist the decision 
makers in allocating federal funds. Second, the number of claimants 
for federal funds is quite large. To quote Mr. Baer, . . . "should 
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the Vietnam war end tomorrow, there would be more than enough 
claimants to spend that money five times over." Third, the decision 
process of allocating federal funds to support various proposed R&D 
programs is usually one in which all of the alternatives appear 
attractive and potentially beneficial . Therefore , it is important that 
the particular relevance of each program and the ways in which antici- 
pated benefits might be achieved be described in detail. 

It was also noted in Section A that this growing requirement 
to define expected benefits and projected costs of proposed computer- 
based information systems is encountering significant difficulties . A 
number of factors appear to contribute to this difficulty . Some of these 
factors derive from the sheer complexity of the system planning and 
development task, and others stem from the system support philosophy 
commonly followed by system planners. Major contributing factors are 
described next. 



An orientation which prevails in the design of computer-based 
information systems is the viewpoint that the information system supports 
the user. In other words, the user is not considered an integral part 
of the information system but rather a recipient of the products or services 
of the system. This common orientation very likely stems from the 
traditional relationship of an individual information user to a library . 
However, this historical relationship has posed a number of knotty 
methodological problems associated with the selection, development, 
and evaluation of an information system; problems which have been 
receiving increasing attention during the past few years. Probably 
the single greatest problem stemming from the traditional relationship 
of the user to the information system concerns the evaluation of a system. 
Some illustrative examples from quite recently published studies should 
highlight this problem. 



i 
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A 1967 report by Wessel and Cohrssen (98) , involving a 
comprehensive literature search and state-of-the-art on criteria 
for evaluating the effectiveness of library operations and services, 
concludes that existing criteria and standards were found to be un- 
satisfactory. This was because only some aspects of libraries lend 
themselves to quantitative measurement, such as number of items 
cataloged, ordered or found in a period of time, whereas the quantitative 
measurement of the value of a library service or product such as a 
lit« 3 rature search, bibliography, or current awareness service seemed 
more difficult to assess. 



In the 1966 Proceedings of the Second Congress on Information 
System Science (78), Ruth Davis noted that the persistence of the evaluation 
problem over a period of 15 years is related to a large number of 
difficulties . Some of these difficulties include lack of well-defined 
objectives, lack of meaningful models, uncertainty concerning measures 
of effectiveness, and lack of quantitative criteria. An article in 1966 
by Parker and Paisley in the American Psychologist (73) pointed out that a major 
problem area requiring research is the development of criterion measures 
for the design and evaluation of information systems . Donald W . King 
in the 1968 Annual Review of Information Science and Technology (22), con- 
cluded that "the literature of 1967, like that of earlier years, is weak 
with respect to cost data .... The general problem of benefits is even 
farther from solution than that of costs." Finally, Marvel Hall in a 1965 
article entitled, " Summary of Study Conference on Evaluation of Document 
Searching Systems and Procedures" (39), remarked that attendees at the 
conference generally agreed that one of the major problems pertained to 
the lack of adequate performance measures and criteria. 



It is proposed that the basic root of this difficulty lies with the 
pivotal concept of "relevance." The general recognition of the centrality 
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of this criterion concept to the system support orientation has led to 
a number of studies and papers within the past few years. A 1965 
study by Human Sciences Research, Inc. (87) on the methodology for 
test and evaluation of document retrieval systems emphasized that a 
better understanding of the concept of relevance is needed. A recently 
completed two-year effort by System Development Corporation (23) 
involving 15 individual studies on relevance judgments suggests that 
relevance judgments can be and are influenced by a number of factors . 
These include the skills and attitudes of the particular judges used, the 
documents and document sets used, the particular information requirement 
statements , the instructions and setting in which the concepts and 
definitions of relevance are employed in the judgments, and the type of 
rating scale or other medium used to express the judgments. The authors 
of the report contend that serious doubt exists with respect to studies 
using relevance scores as stable criteria for system or subsystem 
evaluation in instances where the sources of variation cited above 
have not been recognized and properly controlled . 



In a more pointed vein, in a paper in 1963, Doyle (27) questioned 
the use of relevance as an adequate criterion of system retrieval effective- 
ness. He discussed the possible discrepancy between an individual's 
measurable outward expression of an information need and his real 
information need. A year later, Cuadra (24) argued for the retention of 
relevance as a useful construct while, at the same time, he recognized 
that additional effort is needed to develop meaningful relevance measures. 
However, in a 1968 article entitled "Some Questions Concerning Informa- 
tion Need" , O'Connor (69) noted in the beginning of his paper that the 
questions raised by Cuadra in this 1964 article have not been answered. 
O'Connor continued by identifying three possible meanings of the state- 
ment "satisfy the user's information need" and, for each possible meaning, 
he raised a number of questions which must be answered before the meaning 
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of the concept can be considered sufficiently clear to those who 
advocate its use as a system performance criterion. 

In summary, the current procedure of employing the system user 
as the primary source for system evaluation has raised a number of 
criterion-related problems , and it is unlikely that these problems will 
be quickly resolved. A change in the conceptualization of the role of 
the user to the system may prove to be a more satisfactory approach. 
This approach is discussed in the next chapter of this report. 









A second characteristic of present day specialized information 
systems is the large number of alternative system concepts , configura- 
tions , and components . This stems primarily from the rapid advances 
being made in information system technology. The impact of this 
technology is shown in terms of the rapidly growing number and diversity 
of hardware and software alternatives for performing various information 
system functions and subfunctions . The system planner and the system 
buyer are faced with a large number of discrete hardware/software 
components. In addition, the many ways that these components can be 
combined to form system configurations which, in turn, can be used to 
form a variety of systems , create a most complex cost/effectiveness 
decision problem. These components, subsystems, and system alterna- 
tives vary widely on many dimensions . These include number and types 
of services provided; acquisition, operating, and maintenance costs 
involved; availability of the systems to various numbers and types of 
users; types of information that can be processed and stored, such as 
inventory data, test results, patents, maps, engineering drawings; 
output media such as teletype, TV display, microforms, and printed 
page; and mode of use, that is, current awareness and retrospective 
search and retrieval of information . 
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A third characteristic associated with specialized information 
systems is the diversity of perspectives. The system designer/developer, 
the system operator/maintainer, the system buyer, and the system user 
represent four different ways of viewing an information system. Each of 
these perspectives possesses different sets of requirements, constraints, 
and criteria by which alternative possibilities are generated, considered, 
evaluated, accepted, and utilized. These different perspectives do not 
necessarily belong to different individuals or to different organizations. 
The same individual or organization may be required to view the same 
system from more than one perspective. It is the perspective that is 
the key concept not individuals and organizations. 

The system designer/developer desires to achieve, understand- 
ably, a wide market for his hardware/software components. To reach a 
large number of potential users, he is interested in providing a wide 
range of services at competitive prices. The system opera tor/maintainer, 
on the other hand, is more concerned with minimum operating and 
maintenance requirements, a high degree of system reliability, compati- 
bility with existing facilities and services, and adequate day-to-day 
as well as long termlogistic support for the system. The system buyer, 
from another point of view, is looking for a system which possesses a 
high degree of growth potential, a wide range of services to meet a 
variety of institutional or organizational needs , and an acceptable 
level of both acquisition and operating costs. Finally, the system user 
is concerned with obtaining a high level of system performance, immediate 
availability of the system when required, and capability of the system for 
meeting his information needs as they vary with respect to specificity 
and type. Compromises and trade-offs, of necessity, are required among 
these different perspectives . 

A fourth characteristic of specialized information systems pertains 
to the increasing trend toward evolutionary development. This trend is. 
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in large measure, a function of the rapid growth in information related 
technology. The relatively rapid advances being made in the state- 
of-the-art have produced two major effects which enhance this growing 
trend toward system evolution. First, the advancing technology produces 
a situation in which the next decade appears to hold great promise with 
respect to tremendous improvements in current capabilities and in 

new revolutionary concepts . Because of the complexity and degree of 
sophistication, it generally requires ten to fifteen years to bring these 
theoretically possible concepts into operational reality. The temptation 
is naturally strong to put "one's money" on these future hopes, but, 
the full maturation of these theoretically feasible concepts requires talent. 



money, and time. 



As Melcher (61) points out in an article entitled "Automation: 

Rosy Prospects and Cold Facts, " there are a number of unfulfilled ex- 
pectations in the area of automation. Since the system is not -in -being , 
so to speak, there are little or no empirical cost/benefit data available 
with which to assess the concept objectively, other than what might be 
labelled as "assumption-based" analyses. And the more innovative the 
concept, the less chance there is to find current systems which are 
similar enough in character to use as gross models for estimating the 
cost/benefit characteristics of the future system. The prevailing 
solution to such a situation is to "leam-and-obtain-data-as-the-system- 
evolves , " that is , to support research and exploratory development efforts 
as well as prototype development and production efforts . 

In accordance with this approach, a common strategy is to 
perform these research and development activities within an operational 
or user context to enhance the "validity" of the findings. A major 
question of interest to system buyers is at what stage in the evolution 
of a system concept can a comprehensive and objective basis be used to 
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estimate the probable costs/benefits of the system if and when it 
becomes operational . The earlier in this cycle one can make the 
decision to continue to support or to reject a candidate, the less the 
cost. 



To further complicate the issue, the other end of the spectrum 
is anchored by systems, subsystems, and components which are in- 
being and have accumulated an empirical base of cost/effectiveness 
data, although, perhaps in a different environment. The dilemma is 
whether to "buy" an immediately available off-the-shelf system in 
which cost and performance are fairly predictable, but which promises 
less capabilities, or to cast one's lot with the lesser known but more 
promising future system. With almost unlimited resources, both major 
options might be pursued simultaneously. However, the growing need 
for some solution to the information problem, coupled with limited 
funds, limited talent, and limited time make it necessary for some 
systematic procedure to be developed and used to guide the selection 
of a system from among available alternatives . 

A fifth factor related to this evolutionary characteristic of 
information systems pertains to the integration of new components, 
subsystems, and systems into an on-going information system. 

Consider military systems as one extreme. The proposed new system 
may be constrained only by the nature of the inputs such as documents and 
reports and possibly some of the output requirements such as user loca- 
tions and transportation means. Now consider university -based systems 
as the other extreme. A proposed system might be constrained at all levels; 
that is, system, subsystem, and component, and throughout all of the 
information system functions; including acquisition, cataloging, ab- 
stracting, indexing, extracting, storage, dissemination, and end use. 

In addition, these new systems are introduced into some existing environ- 
ment. This environment involves constraints of various types including. 
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for example , building , storage area , availability of water and 
electricity, and heating and air conditioning. These multiple 
constraints often lead the system planner and designer in the 
direction of either automating existing and highly constrained manual 
procedures rather than tailoring the procedures to meet the new capabilities 
offered by the computer technology, or selecting a system concept which 
is far lesser in scope and capabilities than originally desired. 

The material presented thus far can be summarized succinctly 
in the following five statements . 

. There are limited federal funds available to support R&D 
programs which are growing in numbers and in costs . 

. The primary aim of responsible decision makers is to allocate 
federal funds to those R&D programs which show the greatest likelihood 
of providing significant benefits within established cost constraints . 

. Objective and comprehensive data on projected benefits, 
costs, and risks are needed as inputs to support the allocation process. 

. Experience to date involving the planning, development, and 
operation of computer-based information systems has revealed considerable 
difficulty in providing to the decision makers these needed information and 
data inputs in a timely, accurate, and complete manner. 

. The sheer number of factors and uncertainties involved in 
planning and developing computer-based information systems, coupled 
with the presently prevailing system support philosophy, are judged 
to be the two primary sources of the difficulty experienced. 

A system planning guide, to be potentially useful, must be 
responsive to these two inferred sources of difficulty. As a consequence, 
the approach selected in the conduct of this project reflects this 
orientation . 



C . Approach and Organization of Report 



The approach adopted in developing the planning guide reveals 
four general characteristics . 

First, the level of description achieved In the Information 
system planning guide may be characterized as being macroanalytlc 
In contrast to mlcroanalytlc . There are a number of reasons why the 
broader macroanalytlc approach represented the only feasible alternative: 

(1) the number and complexity of the factors Involved In Information system 
planning would completely prohibit an adeguate mlcroanalytlc treatment 
given the time, skills, and support available. (2) The attempt to substitute 
a task-oriented philosophy for the content-oriented and system-support 
orientation requires an Initial structuring and developmental effort . A 
macroanalytlc approach Is appropriate during this initial formulation 
and definitional phase. (3) This report is primarily management-focussed 
and not technically-focussed. References to selected books and docu- 
ments which describe in detail relevant system analytic tools and opera- 
tional research techniques are noted in appropriate places throughout 
this report. 

Second, an attempt has been made to utilize a few unifying 
constructs as a means of integrating and simplifying the task of 
organizing, screening, evaluating, and selecting system alternatives. 

A graphic construct of the system planning decision process was developed 
to facilitate the integration of major decision components and the interactive 
process associated with the resolution of major system planning decisions. 

Third, the approach taken may be described as being largely 
prescriptive rather than descriptive in nature. The term prescriptive, 
in this context, refers to the use of general guidelines organized into an 
overall system planning structure rather than a step-by-step cookbook 
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concept of prescriptiveness. The goal, in this report, is to achieve 
a level of detail which, on the one hand, is sufficiently broad in scope 
to enable the required creative abilities of the system manager to 
effectively function within a specific system context, and yet on the 
other hand, is sufficiently structured so as to provide the system 
manager with a set of systematic and ordered guidelines to facilitate 
the overall planning and developmental process . 

Fourth, the approach includes both static and dynamic views 
of the system planning and development process. The static view 
includes information on the functional, technical, and administrative 
factors associated with system planning . The dynamic view integrates 
these three parts of a system description into a phase by phase analysis 
of the planning and development process. At each phase, the level of 
system description achievable is identified, the types of decisions re- 
quired are described, and the kinds of inputs possible for evaluating 
system alternatives are noted. 

The remainder of this report is organized into four chapters . 

Chapter II deals with the functional requirements of a system. In 
this chapter, the task-oriented philosophy is described using a grant- 
supported research project as the task to be accomplished by the 
combined scientist/computer-based system. The numerous elements 
which make up a functional description are briefly covered in this 
chapter while the supporting and more detailed considerations of these 
elements are provided in Appendix A to this report. Chapter III is 
concerned with the technical description of a system. A system 
planning model was constructed to facilitate the systematic considera- 
tion of the multiple and varied planning decisions, criteria, and constraints 
found with complex information systems. Chapter IV defines and classifies 
basic administrative functions and identifies management tasks which need 
to be accomplished in establishing a system planning and development 
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organization. Chapter V provides a dynamic description of the system 
planning and development process. This last chapter identifies the 
kinds of functional, technical, and administrative activities occurring, 
and level of system description realizable during each of five system 
planning and development phases. Three types of supporting informa- 
tion are provided in Appendix B. They are the names of individuals 
interviewed during site visits to universities in which system planning , 
development, and operational activities are being pursued; a summary 
of the functional, technical, and administrative aspects of the observed 
information systems , and a discussion of major characteristics associated 
with different information system models . 
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n. Functional Description 



A . Introduction 



The last quarter century has seen a dramatic increase in 
the amount of scientific and technological information.^ Since 1961 
a number of attempts have been made to estimate the volume and 
exponential growth of scientific and technological information ( 2,36, 

45, 52, 72). In a recent article Licklider (51) stated, "The body of 
recorded scientific and technological information now has a volume of 
about ten trillion alphanumeric characters (i.e., letters, numerals, 
and punctuation marks) and is increasing along (what for lack of precise 
data is usually assumed to be) an exponential curve characterized by a 
doubling time in the range 10-15 years ..." Linus Pauling (19) estimated 
the growth in science for the period 1933-1965 by taking the number of 
million words per year published in the Physical Review and since 1958 
the Physical Review Letters . Measured this way over the 32-year 
period scientific research has had an average increase of 7.5% per 
year. A slightly more conservative growth rate was obtained by May 
(57). The results of May's analysis, based on mathematics literature, 
indicate that the well known hypothesis of exponential growth of 
scientific literature is confirmed, but at a rate less than 1/2 of that 
generally found by other investigators; that is, about 2.5% per year, 
doubling about four times a century rather than every ten to fifteen 
years . 



In this context, the term information is used to represent the recorded 
alphanumeric characteristics (that is, letters, numerals, and punctuation 
marks) while knowledge pertains to the output achieved in terms of 
descriptive or predictive utility by systematically arranging, organizing, 
or relating available information items according to various conceptual 
schemes . 



These various estimates all point up the significant growth rate 
of scientific and technological information. To describe this phenomenon, 
such phrases as "information explosion" and "flood-of-information" are 
commonly found in technical and professional articles. However, some 
of the contexts in which those expressions are used seem to imply 
that the dramatic growth in scientific and technological information 
that we are experiencing is somehow the problem to be solved. 

Suggestions have been made to raise journal standards in order to 
reduce the rate of publication, or to hold a moratorium on the publishing 
of current research findings until the publication lag is caught up. These 
suggestions imply that slowing down the rate at which scientific and 
technological findings are reported would help reduce the information 
problem. This viewpoint is inconsistent with the continuing efforts of 
a substantial segment of our society to significantly increase (1) the 
number of trained scientists and engineers in this country and (2) the 
amount of fiscal support to productively carry out basic and applied 
research activities. And, it is certainly incompatible with the pre- 
vailing, although questionable, "publish or perish" philosophy. 

One of the goals of a scientifically and technologically oriented 
society is to optimize the ratio of knowledge generated to information 
generated. Assuming that, under ideal conditions, the ratio is a constant, 
then by increasing the flow or production of scientific information, the 
fund or corpus of knowledge is proportionately increased. The proper 
concern then would seem to be to direct attention to those conditions 
or events which tend to degrade or reduce this "idealized" ratio. 

Viewed in this manner, the problem is the perceived existence of an 
inadequate growth rate of knowledge compared to the growth rate of 
information. Or, to state it another way, it is necessary to direct 
attention to the point in time when research is being planned and con- 
ducted as well as the point in time when findings are being reported 
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and disseminated. This perspective emphasizes the role of scientists 
as producers of information as well as users of information. 

The events or conditions which could adversely affect the 
idealized knowledge/information growth ratio may be quite numerous 
and many of them are perhaps unsuspected. However, if it is accept- 
able to reason backwards from the nature of the corrective actions 
currently being taken as a way of inferring the causal factors, there 
are commonly held to be at least two primary types of concerns associated 
with the growth of scientific knowledge. The first concern is the increasing 
probability that a scientist or engineer will miss relevant work done by 
others and, as a consequence, not be in an optimum position to most 
effectively build onto existing knowledge in a particular area . The 
second concern is the increasing probability that there will be an adverse 
trade-off in time spent by the scientist or engineer in productive and 
creative thinking versus time spent in searching , retrieving , extracting , 
or organizing relevant findings. These conditions could adversely affect 
the knowledge growth rate as compared to the information growth rate. 

The five major kinds of solutions currently being planned, developed, 
or utilized may be grouped under one or the other types of concern expressed 
in the preceding paragraph . Two of the solutions are built around broad 
and heterogenous information bases, that is, library directed solutions, and 
the remaining three solutions are designed around relatively narrow and 
homogenous information bases, that is, specialized information centers. 

The first type of solution is directed at increasing the size of 
the document base through use of networks which connect a number of 
libraries or document sources to user sources . The effect of this approach 
is to make available to users a much larger store of documents than is 
economically feasible within the context of a single library. The network 
solution is oriented, among other goals, to solving the problem of users 
not being aware of relevant information in their field. Examples of 
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regional library network systems are the RICE System at Rice 
University, the BEACON or Academic Library Network System at the 
University of Colorado, and the New England Regional Library 
N etwork . 

A second type of solution is focussed on the problem of reducing 
total system response time through automation. As the number of 
volumes increases through networking or acquisition in document/ 
library systems, there is normally an associated increase in the 
time required to perform the basic document processing functions. 

One major goal of the automation approach is to reduce the amount of 
time the user must spend in searching, locating, and retrieving relevant 
documents . A number of universities are currently planning and developing 
computer-based library systems in which one or more of the document 
processing functions are being automated. Some typical examples in- 
clude University of Chicago, Stanford University, Columbia University, 
Pennsylvania State University, and the University of California at 
Berkeley. 

A third solution involves the collection, reproduction, and 
wide dissemination of scientific and technological titles, abstracts, 
bibliographies , and reports . The larger federally sponsored information 
systems tend to fall into this group. Examples include the Clearinghouse 
for Federal Scientific and Technical Information, the Defense Documentation 
Center (DDC) for Department of Defense contracts. The National Referral 
Center for Science and Technology (NRCST) supported by NSF, NASA's 
Scientific and Technical Information Facility at Documentation, Inc. , 
and the Atomic Energy Commission's Division of Technical Information 
Extension (DTIE) (68,92). The massive reproduction and wide dissemination 
of scientific and technological literature are directed at reducing the 
possibility that relevant research might be missed by geographically and 
organizationally separated groups of scientists and engineers . 
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A fourth solution involves the smaller information systems 
or centers which are specialized or personalized and focus on 
restricted inputs and selected dissemination of information to 
interested scientists and engineers . Representative examples 
include the National Library of Medicine MEDLARS Information Systems 
Division (30) which produces Index Medicus, other recurring bibliographies, 
and demand searches; Mitre Corporation (47) which selectively distributes 
contractual reports to 20 user groups; and the SDI System at Ames, Iowa 
( 102 ) which serve about 200 cases involving the University of Iowa and 
30 diverse industrial firms. The objective of these systems is to i 

maximize the amount of relevant information disseminated to users 
and thus to minimize the amount of total time that might be spent in 

searching, retrieving, and reviewing materials if the user were forced J 

I 

to perform the entire task of screening relevant from irrelevant information. I 

I 

A fifth solution involves a mixture of systems which, however, 
have one major characteristic in common, they provide the individual 
user with special information services . These special services in- 
volve some type of operation on the information contained in documents . 

These special services include extracting "chunks" of information, pre- 
paring special bibliographies , performing reviews , and writing summaries . 

Some of the systems possess a specialized information base, while others 
contain a broad subject matter and discipline base. Some are designed 
around computer components; othere are largely manual in make up. 

Although these systems vary widely in their structural, functional, 
and operational characteristics, at least one of their common goals 
is to actively assist the individual user by performing a range of 
operations on the document- stored information. The desired outcome 
of these special services is to permit the scientist and the engineer to 
allocate a greater percentage of his time to those tasks which require 
more creative and analytical skills and knowledge. Examples include 
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the MACTIP System at MIT (48) which is currently undergoing conversion 
to operational status; Project Intrex at MIT (72) which is in the planning 
stage; the CIS Retrieval System at Lehigh University (102) which is 
conducting on-line computer experiments on the "intellectual processing" 
associated with information search and retrieval activities; the Brain 
Information System (BIS) at UCLA (30); and the Pennsylvania Technical 
Assistance Program (PENNTAP) at Pennsylvania State University. 

Some of the special services provided by these systems are 
similar to what the proposed task-oriented information system would 
provide. For example, the MACTIP System was used by Professor 
Sanborn Brown and a group of his co-workers at MIT to compile a book 
on basic plasma data (12 ). In this instance, the system actively 
participated with the scientists in preparing the book. This example 
illustrates the capability of an on-line computer-based system to func- 
tion jointly with the scientist in producing an output. 

The task-oriented information system approach, like the special 
services solution, is primarily focussed on the second type of concern 
just discussed; that is, the efficient allocation of the scientist's time 
to those research tasks requiring the creative and analytical capabilities 
of man. However, unlike the special service type of solution, the task 
approach is (l) research output focussed rather than service to user oriented 
and, most importantly, (2) planned and developed specifically around the 
requirements and characteristics of phases, tasks, and subtasks associated 
with the grant- supported research projects in contrast to being general 
purpose in nature. The "special purpose" orientation requires the system 
planner to identify and describe, where feasible, the activities associated 
with accomplishment of the task. This planning requirement is discussed 
in the remaining sections of this chapter. 
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B. Task-Oriented Approach 

It is proposed that seven major classes of information are 
needed as inputs for specifying functional requirements. The set of 
functional requirements are needed to guide the planning and design 
of the task-oriented computer-based information system. These 
task-related classes of information include: 

. a general background description of the university-based 
research project, 

. a detailed analysis of the phases, tasks, and subtasks 
generic to grant supported research projects, 

. an identification of the major variations in research project 
characteristics as a function of differences in subject matter, 
research objectives, and methodology used, 

. a grouping of the various research project phases, tasks, and 
subtasks into a few discrete categories characterized by a 
high degree of commonality in terms of their performance or 
functional requirements , 

. the development of appropriate product (benefit) criteria 
and identification of relevant criterion sources for evaluating 
research project outputs, 

. a determination of existing attitudes and behavioral 

characteristics of the selected scientist population regarding 
the research project performance requirements, and 
an Identification of environmentally originating contributions 
and constraints which are relevant as inputs to system 
planning . 

It is beyond the scope of this study to accomplish the required 
complete, and detailed functional analysis of grant-supported research 
projects. Rather, the preliminary development of a research structure 
is presented with sufficient accompanying description to facilitate an 
effective level of communication which conveys the essence of this 
task-oriented approach. Extensive use is made of graphic illustrations 
as a means of identifying the kinds of data and information needed to 
accomplish a complete functional specification. 



II-7 



C. General Description 



Figure 1 shows a schematic representation of the Research 
Project (R-P) System located within the university environment. The 
major points illustrated in the figure are given below, and their 
significance with respect to the task-oriented approach is discussed 
immediately following . 

. The planning, conduct, and documentation of a grant 
supported research project represents the mission or purpose of the 
R-P System. The grant supported research project has an externally 
definable beginning (a proposal) and end (a final report) . Like a 
number of other goal directed and sequentially dependent activities , 
a research project can be meaningfully broken down into phases, tasks, 
and subtasks . Each of these parts can be identified and described with 
varying degrees of confidence and detail in terms of input, process, and 
output characteristics. It is useful to view the subtask, task, and phase 
outputs as sub-goals of the system. The final or overall output/goal is 
ideally a completely and accurately documented research report which is 
of timely scientific value. 

. The components of the R-P System, like other systems, consist 
of people, hardware, and software. The R-P System which performs the 
various research project activities is composed of two major subsystems; 
i.e., a scientist subsystem and a service subsystem, which perform 
each of the research project subtasks separately or jointly. Even when 
the two subsystems are processing subtasks independently, however, 
they must coordinate the inputs and outputs of the various sequentially 
related subtasks. 

. The two major subsystems may be considered as independent 
subsystems . That is , they perform activities other than those directly 
related to the accomplishment of a research project. Here we are 
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concerned only with those activities which directly contribute to the 
accomplishment of the research project. 

. The grant supported research projects of interest to this 
study are performed within the university environment. The university 
environment may be viewed as consisting of several organizational 
structures designed to fulfill the numerous and diverse functions of a | 

university. These major functional entities are educational, administrative, 
research, athletic, counseling, and social in nature. The two university 
based functions of primary interest here are basic research and what might 
generally be labelled as project administration. Project administration 
includes those individuals or time, facilities, and procedures within ; 

the university organization which are directly concerned with the 
administrative and logistical support activities associated with the pro- 
curement, administrative monitoring, and contractual completion of 
externally funded basic research projects. The basic research function 
is performed by the scientific and professional segment of the university 
which proposes , plans, conducts, supervises, and evaluates basic 
research. These scientists generally have available for their support 
some technical facilities, student assistance and scientific litera- 
ture, and professional colleagues who serve as sounding-boards 
for their ideas and research-related findings. 

. Beyond the immediate university context there exists two 
important entities, the scientific community as a whole and the agencies 
or organizations which support, through grants, basic research in the 
university environment. In a sense, there are three relevant contexts 
concerned with grant supported research projects. The most immediate 
context is the R-P System itself, and, specifically, the scientist 
subsystem and the service subsystem. A broader context consists of 
the university environment and particularly the project administration j 

and professional or departmental colleagues . The broadest context 
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includes the sponsoring agencies and those members of the overall 
scientific community whose discipline and problem area interests overlap 
those of the particular university based scientist in question. These 
three contexts and the six major entities within the three contexts 
represent the major sources of requirements, constraints, and criteria 
for generating and evaluating alternative R-P System concepts . 

The proposed concept of a R-P System shifts the traditional 
relationship of the user to the system. Rather than considering the scientist 
as the recipient or user of the services or products of an information 
system, the scientist and the service jointly contribute as producers 
to the research project outputs. The accomplished phases, tasks, 
subtasks constitute the system's outputs. Also, the so-called user 
of the research project system has grown to include interested university 
members and the professional community as a whole. By shifting the 
relationship of the scientist to the system and by expanding the meaning 
of the term user, some of the major traditional evaluation problems have 
been by-passed. The research project outputs of the system can be 
evaluated independently by departmental colleagues, professional 
journal editors, and the professional community, as a whole, as well 
as by the individual scientist. Scientific methodology, professional 
journal requirements, and cognizant scientists in the problem area of 
concern provide a rish source of standards that can be applied to the 
research product outputs . 

. The use of the research project model as the foundation for 
the R-P System serves to combine a functional or task-oriented scheme 
with the more traditional content-oriented information system scheme . 
System generated inquiries are concerned with fulfilling not only specific 
information needs but also information needs for some specific purpose or 
task. One may visualize, for example, a three-dimensional matrix 
consisting of a functional dimension which lays out the phases, tasks. 



and subtasks of a research project/ a content dimension which organizes 
material with respect to its subject matter, and a procedural dimension 
which consists of computer programs or specified manual procedures 
for processing particular content -bearing units of information needed to 

i 

accomplish some specific subtask. For example, a computer program j 

may be developed for editing grant proposals. The basic program may 
permit variations to allow for differences in subject matter or in potential 
sponsoring agency requirements. The potential ability to specify subtask 
outputs, subject matter inputs, and processing requirements represents i 

a shift from the more restricted emphasis on subject matter inputs. 

. Use of the research project model as the basis for the R-P 
System extends the scope of an information system to include all of 
those activities associated with the accomplishment of a research 
project. The R-P System will not only perform the information retrieval 
function but will also perform such functions as computation, simulation, 
editing, abstracting, and controlling of stimulus presentation and recording | 

of responses of subjects during experimental trials. It will encompass the 
full range of capabilities, services, and products currently being explored, 
used, or developed in existing computer-based systems. 

D. Research Project Phases , Tasks, and Subtasks 

’ Figure 2 shows an incomplete schematic representation of the 

}. 

research project phases, tasks, and subtasks. The grant supported 
research project is made up of six phases: Si proposal preparation 
phase, a general planning phase, a detailed planning phase, a data 
collection phase, an analysis and interpretation phase, and a report 
preparation phase. Within each phase, there are tasks normally 



associated with the accomplishment of the particular phase. And, 
for each task, there are subtasks to be performed. A preliminary 
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version of such a research project model is presented in Appendix A 
of this report. The model is intended to be a comprehensive and generic 
representation of the phases, tasks, and subtasks associated with the 
conduct of grant supported research projects. The model is designed to 
encompass the full range of activities that might be performed during 
grant supported research projects, although the performance of any 
given research project may not require the execution of all tasks and 

subtasks listed. 

E . R"P System Performance Requirements 

A necessary task to be performed during the early planning 
stages of any system is the specification of the nature and level of 
performance that will be required to achieve the objectives of some 
specified job or mission. The tasks and subtasks of the research 
project model represent sub-objectives or sub-goals to be accomplished 
during each of the project phases. Accomplishment of each sub-objective 
or sub-goal requires a certain level and type of performance . System 
components are selected or designed to cost/effectively perform these 
tasks and subtasks in order to achieve the stated sub-objectives. 

In this report, an initial step has been taken towards the state- 
ment of research project performance requirements. A general classifica- 
tion scheme has been developed. This preliminary scheme reflects an 
initial judgment concerning the types of skills and knowledge required 
to perform each of the research project tasks and subtasks. It proposes 
that three broad groups of skills and knowledge are brought into play 
during the conduct of a successful grant- supported research project. 

The three classes of skills and knowledge are scientific, managerial, 
and informational in nature* 
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Scientific skills and knowledge refer to technical and creative 
capabilities. Technical skills and knowledge pertain to both subject 
matter content and methodological abilities acquired by the scientist 
during his formal education and during the subsequent period of time 
when he is a teacher and researcher in professional/scientific 
environments. Creative skills are much more difficult to define, 
however, there are four types of events in which the term creative 
thinking is commonly applied. First, the early perception or diagnosis 
of a problem is one type of event. The ability to perceive a deficiency 
in an existing theory and the recognition of a deficiency in a current 
technique or method are examples . A second type of event is the 
realization of the importance or significance of a finding or occurrence, 
although the discovery may have been of an accidental nature . A third 
is the ability to develop a more encompassing theory than an existing 
one which will account for more of the phenomena in a particular problem 
area. The development of a more effective technique or method falls 
into this category. A fourth type of event is the ability to perceive 
that knowledge or a technique developed in one area can be applied 
to another type of problem often in an entirely different area . 

Managerial skills include resource management and clerical 
skills . Resource management refers to the cluster of skills and 
knowledge related to the planning , allocation , coordination , and 
supervision of both human and material resources . Activities such 
as scheduling , allocating personnel to various research tasks , 
matching projected research effort to money requirements , and effective 
utilization of facilities and space represent examples of resource 
management. Clerical skills include such activities as typing, editing, 
transmitting materials, coding, filing, and organizing materials. 

Informational capabilities fall into an area which overlaps 
both the scientific category and the managerial category. Some of the 
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specific activities associated with information handling, such as 
acquisition, compilation, and organizaticn of chunks of information, 
emphasize requirements for both clerical and resource management 
skills. Other specific activities associated with information process- 
ing require technical and perhaps even creative skills and knowledge. 
Nevertheless, from a system planning and development point of view 
it is useful to separate this category from both the managerial and the 
scientific categories . The informational category provides a transition 
between the traditional concept of specialized information services for 
scientists and the proposed concept of a research project model. Further- 
more, the informational category is restricted to those research activities 
involving information obtained about the ideas, work, and findings of 
other scientists. For example, a number of the tasks and subtasks clearly 
emphasize the retrieval and organization of externally originated informa- 
tion, that is, review, screening, and extraction/tagging of relevant 
chunks of information found in the literature and the review of literature 
for discussions of relevant variables and their characteristics. The 
hypothesized type and level of performance capabilities required through- 
out the planning, conduct, and documentation of a research project 
are illustrated in Figure 3 . This figure presents an obviously over- 
simplified picture. Extensive, analytical, and empirical efforts are 
required to factor the tasks and subtasks during each phase into 
differentially weighted skill and knowledge requirements . 

F. Research Project Paradigms 

Another requirement in system planning is to achieve an 
operational description of the selected task. The description should be 
both comprehensive and yet sufficiently detailed to serve as an effective 
input to system-based decision making. In other words, the description 
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Figure 3 

Level of Skill and Knowledge Requirements 
for Specific Phases of a Research Project 



Skills and Know- 
^''^^Jedge Require- 

ments 

Phases 


Scientific 


Informational 


Managerial 


Phase I 

Proposal Preparation 


High 


High 


Low 


Phase n 

General Planning 


Low 


Medium 


High 


Phase III 

Detailed Planning 


Medium 


Medium 


Medium 


Phase IV 
Data Collection 


Medium 


Low 


Medium 


Phase V 

Analysis & Interpretation 


High 


High 


Low 


Phase VI 

Report Preparation 


Low 


Low 


High 
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should not produce such a voluminous amount of descriptive material 
that either intellectual comprehensiveness or meaningful manipulation 
of the data is prevented. 

One strategy for helping achieve this balance between depth and 
breadth of coverage is the grouping of large numbers of task-related 
activities into relatively few categories. It was stated above that one 
of the objectives is to develop a list of research project phases, tasks, 
and subtasks common to most grant-supported research projects. A 
preliminary effort to develop such a generic list revealed that some tasks 
and subtasks are common to most, if not all, grant- supported research 
projects; particularly during Phase I Proposal Preparation and Phase VI 
Report Preparation. This was found to be less true during Phase II 
General Planning and Phase V Data Analysis and Interpretation, and even 
less true during Phase III Detailed Planning and Phase IV Data Collection. 



Variations found among grant-supported research projects may 
stem from a number of factors . Factors which may significantly influence 
types of research project tasks and subtasks undertaken include differences 



in research objectives - uncovering the existence of a particular phenomenon 
versus determining functional relationships among two known phenomena; 
discipline or subject matter - physical matter versus biological organisms; 
methodology - experiment versus controlled observation; scale of measure- 
ment - ordinal versus ratio; and form of explanation - functional or 
teleological versus deductive. Given this range of potentially significant 
factors, the ideal objective is to select or develop a classification 
scheme which achieves a comprehensive coverage of diverse types 
of research projects while requiring only a small number of operationally 
meaningful categories to do the job. A number of possible schemes have 
been explored in a very preliminary manner. 

One classification scheme involves the paradigm model. T. S. 

Kuhn (50) in his book entitled. The Structure of Scientific Revolutions 
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introduced the term paradigm as a label for grouping together laws , 
theories, applications, and instrumentation associated with some 
particular scientific achievement. These paradigms or schools-of- 
thought tie together a community of practitioners who are provided 
with a model, problems, and solutions. These are the traditions 
which the historian describes under such rubrics as "Ptolemaic 
astronomy (or Copemician) , "Aristotelian dynamics" (or Newtonian) , 
"Corpuscular optics" (or wave optics), etc. The significant point 
is that a paradigm represents a closely integrated way of thinking 
about and doing scientific research. However, Kuhn points out that 
the existence of paradigms is restricted to the more mature natural 
sciences. The social sciences and most of the biological sciences 
are currently in the pre-paradigm phase of evolution. While the 
paradigm model does not encompass the full range of disciplines to 
which this study is addressed, it may prove quite useful as a model 
for specialized systems which support a well defined community of 
practitioners in the natural sciences , such as high energy physicists . 

Another alternative classification scheme is based on the 
nature of the subject matter; that is, physical, biological, and social 
subject matter. In support of this alternative, F. J. Ayala (3 ) in 
a recent article argued for the scientific autonomy of biology based 
on the unique functional characteristics of living organisms . These 
unique functional characteristics, according to Ayala, are not reducible 
to physics as fostered by some philosophers of science. Teleological 
explanations are an integral part of biology and living organisms in 
contrast to physics and physical matter. In a similar vein, J. Jaynes (43 ) 
contrasted the basic nature of physics and psychology. Jaynes postulated 
that there are many routes to science rather than a unity of science. He 
states, "My point is that the history, philosophy, and sociology of one 
science should not be modeled on that of another, that there is no such 
thing as normal scientific progress, no one pattern of scientific activity, 
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no one criterion of excellence though there may be of aesthetic 
satisfaction/ that there is no one "scientific" method/ and no one 
way of scientific history." Recognizing that the subject matter of the 
biological and social sciences as well as the physical sciences would 
have a significant influence on the planning and execution of a research 
project/ one problem in developing a useful classification scheme is to 
dp^ermine where to establish the boundaries. In the same article/ Jaynes 
reviewed a recently published book by Hermstein and Boring , 
entitled Source Book in the History of Psychology. Hermstein and Boring 
pointed out in their book that there were fifteen separate tracks usually 
going in different directions / ranging from Fechner's psychophysics to 
Kohler's gestalt psychology. In addition to the problem of establishing 
boundaries between different tracks/ pursuing this alternative across the 
full spectrum of disciplines would very likely lead to such a large 
number of categories as to be unmanageable. By way of contrast/ 

Siever (86) in an article entitled/ Science; Observational/ Experimental/ 
Historical argues that there is only one kind of science/ although there 
are many styles. He feels that little use will be attained by emphasizing 
stylistic differences among the various disciplines and methods of research. 

A third possible classification sche.uts hypothesizes that there 
is a natural clustering of subject matter content/ research objectives/ 
and methods . That is , investigators will select a method which best 
fits the content and objective. As a preliminary check on this hypothesis/ 
a sample of approximately 60 articles published in various journals 
covering physics / chemistry/ biology/ and the behavioral sciences were 
reviewed/ and the objectives/ methods/ and content were extracted and 
summarized. Although there was some evidence of patterning or clustering/ 
the findings were not sufficiently unambiguous to provide a clear-cut 
guide for developing a classification scheme based on the clustering 
hypothesis . Details of this preliminary effort are presented in Appendix 
A of this report. Although this exploratory effort did not uncover the 
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hypothesized clustering phenomena in a sufficiently definitive form, one 
useful finding did emerge . The particular methodology used by the 
scientists dictated, to a large extent, the particular tasks and subtasks 
involved in the research. While not completely adequate as the basis 
for a classification scheme, it was decided to tentatively pursue the 
methodological scheme. The articles reviewed showed that the 
methodology used by the investigators could be generally grouped into 
one of four classes: experimental, statistical, observational, and 
modeling . Figure 4 graphically shows the degree of commonality 
hypothesized to exist among the four methods during the six phases 
of a grant-supported project. 

In a controlled experiment , the researcher manipulates , within 
specified limits, certain features in a situation which are assumed to 
constitute the relevant conditions for the occurrence of the phenomena 
under study. Manipulation and reproducibility "at-will" are the two 
major character! tics of the experimental method. The statistical method 
is characterized by the manipulation of numerically assigned descriptors 
of various events or objects . Searching for trends or functional relation- 
ships is a common objective found with the statistical method. Historical 
facts , sociometric information , and meteorological data are frequently 
subjected to the application of the statistical method. Controlled 
observation represents a deliberate search for contrasting occasions 
in which the phenomenon is either uniformly manifested or manifested 
in some cases but not in others. Astronomy serves as a classic 
example of a science which relies primarily on controlled observation. 



Also, social sciences and some specialties within the biological sciences 
frequently use the controlled ooservation method. Modeling, in this 
context , refers to the construction of either a functional (mathematical) 
or structural (physical) model to depict the "true" nature of some 
phenomena of interest. Modeling is predominately found with the mat"*‘“ 
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Figure 4 

Degree of Commonality Among Research Project Paradigms 
Tn Terms of Task and Subtask Descriptions 
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physical and chemical sciences. Success in modeling depends on the 
existence of a wealth of. related findings and facts on which to build the 
model. A recent example of a most successful structural model is the 
construction of the double helix to depict the make-up of the DNA 
molecule (97 ) . 

G. Objective and Criteria 

The preliminary formulation of an overall objective of a grant- 
supported research project was guided by four considerations . First, 
the objective should reflect the essential purpose of basic research 
within the domain of science. Second, the stated objective should be 
capable of being analyzed into operationally measurable concepts . 
Third, the stated objective should be consistent with the perspectives 
and/or goals of those who would have a legitimate and potential 
interest in the products of grant- supported research. This would 
specifically include the six entities mentioned earlier: the scientist 
and service subsystems, the university based project administration 
and departmental colleagues , and the outside scientific community 
and agencies which sponsor basic research. Fourth, the stated 
objective should be sensitive to the scientific information explosion. 

A preliminary definition of the objective of a grant- supported 
research is as follows: "the overall objective of a grant- supported 
research project is to add to our existing store, scientific knowledge 
of wide application or generality , in a problem area of active intere^ 
to a segment of the scientific community. The three key phrases in 
this definition are to add to . of wide appl ication or generality , and 
nf active interest ." The next three paragraphs briefly discuss these 

key phrases . 
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Scientific knowledge may be characterized as an edifice; that 
is, a large, highly interrelated abstract structure. Symbolically speaking, 
to add to the existing edifice, it is necessary to be aware, at some 
selected point in this abstract structure , of the existing knowledge 
and of the relationships among the related elements of knowledge. 

The exponential increase in recorded scientific information discussed 
earlier has led many individuals to question human intellectual ability 
and interest to adequately understand that which is already known in 
a particular problem area such that the individual scientist can do 
productive research at the frontiers of a particular problem area (104). 

The assumption underlying this position is that failure to be cognizant 
of all relevant information concerning the problem area decreases the 
likelihood that the scientist will add to existing knowledge. 

The second key phrase of the definition is of wide application 
or generality . This part of the definition is concerned with the value 
or significance of the grant- supported research project. More specifi- 
cally, it is concerned with the inherent importance of the findings 
generated during the conduct of a research project. At least four types 
of research findings would have wide application or generality. One 
type of significant research finding might be labelled as the historical 
first. Included under this category would be the full range of discoveries 

' t 

involving natural phenomena . A second important finding is the replacement 
type. Theories which replace other less encompassing or less exacting 
theories and techniques of observation or measurement which replace 
less precise or reliable techniques are in this category. Statistical 
as well as experimental techniques and methods would be included here. 

The third type is that which possesses social significance. Medical 
and nuclear research both contribute, albeit in different ways, a large 
percentage of findings which have social significance. Findings from 



11-24 



■Hi 












medical research on cancer, heart disease, and aging possess 
obvious social significance. Likewise, advances made in 
controlling nuclear sources of energy would have both economic 
as well as social importance. The fourth type of significant 
research finding is that which is transferable. That is, an item 
of knowledge , a method , or a technique developed in one discipline 
may be transferred and effectively applied in another discipline or in 
3 quit© different problem area within the same discipline . The increasing 
trend towards interdisciplinary research has accentuated this type of 
activity . 

Finally, science is dynamic in nature and areas of general 
interest are continually shifting . One factor significant to grant- 
supported research is the timeliness of the research, that is, the 
extent to which the proposed research deals with a problem of interest 
to at least a segment of the professional community and to one or more 
of the grant- supporting agencies. A general assumption is that the 
greater the extent that the proposed research treats problem areas 
of current interest , the greater the likelihood that the research will 
be funded and the findings awaited with interest. Timeliness pertains 
not only to the proposal preparation , the planning , and the conduct 
of the project, but also to the documentation and dissemination of the 
findings . 

Using the preliminary definition of the objective of a grant- 
supported research project as the point of departure, the next step 
is to derive criterion concepts. For example, the term "relevance" 
discussed earlier in this paper is a criterion concept. It embodies 
the notion or idea that retrieved documents or extracted items of 

■j 

information will meet with varying degrees of success , the information 
needs of a user. Likewise the task-oriented approach requires the 
selection of appropriate criterion concepts which reflect the objective 




of a selected task. In this instance, the planning, conduct, and 
documentation of a grant -supported research project is the task to be 
accomplished; and, the preliminary definition stated above is the 
objective of the task. Criterion concepts are used to evaluate the 
completed research projects and, by inference, the R-P System, 
that is, the scientist and service subsystems which produced the 
research. Once suitable criterion concepts are identified, the next 
step is to select valid, reliable, and administratively feasible ways 
of operationally defining the criterion concepts . For convenience , these 
operational definitions are labelled as criterion measures. Strictly 
speaking, however, a full definition of a criterion measure would 
include the event or property of an object to be measured, how the 
measurement is to be performed, and under what set of conditions. 

Figure 5 lists three criterion concepts and some alternative ways of 
operationally defining these criterion concepts in terms of criterion 
measures . 




The criterion concept of adequacy refers to the capability of an 
R-P System to generate a research project which adds new knowledge. 

It is assumed that the extent to which the system produces a research 
report or article which meets the technical, administrative, and 
professional requirements of scientists and journals is a measure 
of its ability to add new knowledge. The editors and referees associated 
with the various professional journals serve as the criterion source for 
assessing adequacy. Adequate coverage of relevant literature, a 
logical formulation of the hypothesis, the quality and completeness 
of the detailed planning, and numerous other factors are examined 
when a completed research study is being considered for possible 
publication. Over the past few years considerable debate has been 
directed at the level of standards exercised by the editors and referees 
of professional journals. Many individuals feel that their standards 
should be raised. In essence, this means increasing the value(s) 
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Figure 5 . Criterion Concepts and Alternative Measures 
of the Research Project 
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assigned to the acceptance -for- publication criteria so that approval 
for publication is given to only high quality research. Although the 
question of journal standards is certainly relevant, the criterion 
concept of adequacy is one of degree rather than a go/no go concept. 

In other words, as the standards are raised, those studies accepted 
for publication should be more adequate . As the degree of adequacy 
increases, then the probability of adding new knowledge to that 
already existing increases accordingly. 

Survivability is a term selected to indicate the adjudged value 
or worth of the findings obtained on a grant- supported research project. 
Some possible criterion measures include the number of different journals 
in which the article is published, the number of different authors who 
cite the research, whether the author receives professional awards, 
and the number of different requests by scientific and lay organizations 
and groups for special presentations by the author. 

The concept of pervasiveness was selected to reflect the 
degree of scientific interest associated with project content. As 
noted earlier, certain problem areas are of high interest within the 
scientific community or a segment of it . The extent to which the 
research project system is concerned with these high interest areas 
should be measurable in terms of such criteria as total number of 
requests for pre-prints, an upward shifting in the priority of publica- 
tion date, and number of requests for a paper given at a convention. 
Extending the concept of pervasiveness somewhat, it is reasonable 
to assume that if the nature of the problem being investigated 
possesses social significance as well as scientific interest, it is 
likely to receive even greater attention, not only from the scientists 
but also from the public in general. 
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The three criterion concepts require the system to be operational 
or adequately simulated in order to obtain the necessary criterion” 
related information . Survivability represents the long term criterion 
concept. A substantial amount of time is required to collect data 
bearing on this criterion. Pervasiveness may be considered a mid- 
range criterion. To estimate the degree of pervasiveness, the research 
product must be widely disseminated to users, and sufficient response 
time must be permitted. Adequacy is the short-term criterion concept. 
Journal editors, sponsors, and departmental colleagues should provide 
a relatively quick indication of the adequacy of the research product. 

Within the context of the research project cycle, different 
criterion sources can be related to the different phases of the research 
project. Figure 6 summarizes primary criterion perspectives or sources 
for the various phases of a research project as well as for the completed 
product . 

H. Scientist Attitudinal and Behavioral Characteristics 

Thus far, attention has been directed to the need for identifying 
and describing activities and performance capabilities required during 
the planning, conduct, and documentation of a grant-supported research 
project. The problem of developing a generic list of tasks and subtasks 
has been discussed. Types of capabilities needed to effectively produce 
a complete research product have been identified, and a preliminary 
definition of the objective of a grant- supported research project and 
three criterion concepts for use in evaluating the quality and timeliness 
of generated research reports and papers have been introduced. Now the 
focus of attention shifts from the research project itself to the scientist 
who is responsible for performing the research and to the context or 
environment of the research project. To be consistent with the orientation 
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Figure 6 

Primary Criterion Perspective/Source 
During the Research Project Cycle 



Research Project Phases 


Primary Criterion Perspectiye/Source 


I. Proposal Preparation 
(and Final Product) 


Sponsor and Professional 
Community 


n. General Planning 


Uniyersity Administration 
and Department 


III . Detailed Planning 

IV. Data Collection 

V. Analysis and Interpretation 


Scientist and 
Colleagues/Staff 


VI. Report Preparation 

(for Publication and/or 
Presentation) 


Journal Editors/Referees 
and/or Conyention Staff 
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adopted in this report, both the scientist and the supporting services 
should be treated together since the so-called R-P System is composed 
of both. However, because of the prime «importance of the scientist in 
the R-P System and because historically the scientist and the supporting 
services are separate , the latter will be treated with environmental 
considerations . 

Knowledge of project-related attitudes and behavioral character- 
istics of university scientists is critical as an input to system planning 
for at least four reasons. First, information about scientists' attitudes 
serve as a major source of constraints in system planning . The decision 
task of allocating research project functions to either the scientist 
subsystem or the service subsystem, or to both for joint accomplishment, 
is significantly influenced by the motivations and desires of the 
scientists themselves. The scientist, for political and professional 
reasons, may choose to perform certain research functions even though 
the performance of these activities is within the technological state- 
of-the-art and is economically feasible. Second, for those research 
project functions which have been assigned to the scientist and service 
subsystems for joint accomplishment, it is important for system de.^ign 
purposes to be thoroughly cognizant of the behavioral patterns shown by 
scientists in performing the particular research activities involved. The 
design of the service subsystem should be based on optimizing the dynamic 
interactions between the scientist and the service subsystem during the 
conduct of required research tasks and subtasks. Third, for those 
research project functions which will be accomplished independently 
by the scientist and service subsystems , the structural and functional 
input/output interfaces are critical design factors. Both attitudinal 
and behavioral characteristics are important in guiding the design of 
input/output components and procedures. Fourth, although the service 
subsystem should play a dominant role in the performance of a research 
project, the scientist has traditionally used a number of formal and 
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informal sources of information and supporting services . Whether 
it would be efficient to shift the full supporting responsibility to the 
university-based service subsystem is questionable. In any case, 
it is necessary to identify these various sources and services and to 
describe how and under what conditions scientists utilize them dunng 
the conduct of a grant- supported research project. Information of this 
nature will help establish the scope and boundaries of the service sub- 

system. 

A review of relevant literature on the research-related 
behavioral characteristics of scientists shows a rapid increase 
during the past few years concerning scientist information needs 
and uses. A gross measure of this increasing emphasis on the study 
of scientist information needs and uses is shown in Figure 7 . 

Menzel, in Volume 1 of the Annual Rev iew of Information Science an^ 
Technology (20), summarized the findings of 23 studies which dealt with 
the topic of scientist information needs and uses . The literature spanned 
the period 1963 through 1965. Saul and Mary Hemer (1967) in Volume 2 (21) 
cited 38 studies covering, for the most part, 1966. Paisley (22) cited 
68 relevant studies covering primarily 1967. In spite of variations in 
length of time actually covered in the Annual Reviews , and possible 
differences in the review thoroughness of the authors, the size of the 
differences shown in Figure 7 suggests that there is an increasing 
interest in the problem of determining scientist information needs and 
uses as inputs to system planning . Along with the attention being 
given to the problem, there has been a strong trend toward improving 
the methodology employed in studying the information need problem 
and a beginning trend toward developing an adequate conceptualization 
of the user and his information needs . 

The wide ranging studies on scientist information needs and 
uses have utilized questionnaires, direct observation, interviews. 
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diaries, laboratory experiments, and on-line information systems to 
investigate information needs and uses. University, non-profit 
organizational, laboratory, and industrial settings have been used. 

The types of behaviors studied have included research project related 
behaviors, characteristics of personal indexes, oral/informal technical 
communications, and behavioral characteristics associated with formal 
sources and types of information (20, 21, 22, 26, 31, 34, 35, 42, 54, 
62, 75, 76, 77). 

In reviewing the literature on information needs and uses, two 
related observations seem relevant. First, although increa:»ing interest 
and methodological sophistication is being shown in the problem area, 
and the amount of data and findings being generated is increasing, it 
is not clear just how the system planner can use this growing wealth 
of information in helping to design an information system. It is 
assumed that a primary, if not the sole, objective of most of these 
studies is to contribute to system design. If so, it would seem that 
a great deal of effort would be made to show specifically how the 
findings should influence or at least be relevant to system design. 

As Van Cott and Kinkade (95) point out in a recent report there is a 
need for bridging the gap between so-called user studies and system 
design requirements studies. 

A second and related observation concerns an apparently 
common assumption that to achieve valid data for system planning 
and development purposes, it is necessary that the study be under- 
taken using real or prototype systems capable of meeting the informa- 
tion needs of real users functioning in a realistic environment. This 
assumption seems to be rather commonly held by those involved with 
computer based information systems. However, the high cost involved 
in using fairly large prototype or experimental test beds for research 
purposes raises a point for consideration. If specific system design 
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related questions covering information needs and uses can be clearly 
articulated, we may find that the answers to a significant number of 
these questions can be obtained through less expensive means . 

And, even though there may result a general consensus that real or 
prototype systems represent the optimum setting, the task of clearly 
identifying and articulating the design-related questions is an important 
requirement in its own right . 

Figure 8 illustrates, at a very gross level, how findings 
concerning research related attitudes and behaviors of scientists 
could have system planning implications . Inspection of Figure 8 
suggests that primary responsibility for performing research tasks 
and subtasks requiring scientific capabilities will be done largely 
by the scientists or jointly by the scientist and the subsystem. This 
will undoubtedly be the case with those tasks requiring creative abilities, 
for example, developing hypotheses and designing data collection equip- 
ment techniques. On the other hand, it is reasonable to assume that a 
number of the straight forward tasks , for example , computational tasks , 
can be allocated to the service subsystem for accomplishment . With 
respect to those tasks requiring information retrieval and the organization 
of information, the service subsystem likely will be assuming a greater 
degree of responsibility as the size of the pool of relevant information 
increases , and concomitantly as the ability of the scientist to keep 
abreast of the relevant literature "on his own" decreases. 

Finally, the scientist might willingly leave to the service subsystem 
the tasks requiring both clerical and resource management skills and 
knowledge . 
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Figure 8 

Scientist Attltudlnal and Behavioral Characteristics 
With Respect to Research Prelect Requirements 
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I. Environmental Constraints and Contributions 



The scientist and his research-related attitudes and behaviors 
have been considered . Now the attitudes and behaviors of various 
environmental sources as they pertain to the research project will 
be considered. These environmentally originating attitudes and 
behaviors are translatable into system planning constraints , require- 
ments, and supporting services. 

As pointed out in the preceding section, while an increasing 
amount of attention is being devoted to the scientist, at least with 
regard to his information needs and uses , there is little known about 
the viewpoint of the environment in which the scientist functions . With 
the notable exception of current interest in the question of federal funds 
tor basic research, searching the available literature has turned up 
little information directly relevant to this question. 

Nevertheless, the importance of the research environment 
is well recognized as shown by the information obtained during visits 
to universities where computer-based information systems are being 
developed. See Appendix B for a summary of the information obtained 
during visits to the selected universities . Comments of individuals 
interviewed during these visits highlight the criticality of the environ- 
ment. In reports which discuss scientist information needs and uses, 
statements about the environment also point up this criticality. As an 
example. Paisley (22) in the 1968 Annual Review of Information Science and 
Technology identified eight different external contexts that are critical 
with respect to the scientist information needs and uses . These eight 
contexts include the scientist within his work team, within a formal 
organization, within an invisible college, within his reference group, 
within a membership group, within a formal information system, within 
a political system, and within his culture. However, these different 



external contexts are usually viewed from the perspective of scientist 
rather than from the perspective of the environment. 

Figure 9 illustrates the role of the environment as a source of 
system planning constraints, requirements, and research support. 

The information contained in the cells is largely conjectural, like 
the information in Figure 8, but it should help as a vehicle of 
communication . Environmental sources are grouped under university- 
based entities and entities external to the university. Within the 
university environment, there exist facilities for supporting research, 
professional faculty and staff, and the administrative officers and 
staff. Within the external environment, the research project sponsor 
is included, and journal editors and staff members for professional 
conventions, and scientists/engineers who are members of the general 
professional community are also included. The column headings depict 
the major kinds of skills and knowledge required to plan, conduct, and 
document a grant -supported research project. The information presented 
in Figures 8 and 9 are intended to convey two major points. First, 
the so-called attitudes and behaviors of the scientist and the environ- 
mental agencies are directed to a common object; that is, the research 
project functions or requirements. It is these research project functions 
which will be allocated to the scientist or service subsystem, and it is 
these functions around which the system will be planned and designed. 
Second, the different environmental agencies provide the system planner 
with a source of system planning constraints, requirements, and support. 
The service facilities, the professional faculty/staff, and the professional 
community provide administrative and technical support for the research 
project. The university administration provides constraints in terms of 
policies and procedures . The sponsor provides support in the way of 
funds and information and requirements , and the journal editors/ 
convention staff provide requirements of a scientific, informational. 
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Environmental Constraints and Contributions 



Research 
Project 
Require - 
ments 

Environ- 

mental 

Sources 


Scientific 


Informational 


Manaaerial 


University Environment 


Service Facilities 


Provides compu- 
tational and 
laboratory- 
related services 


Provides library- 
related services 


Provides clerical 
services and 
administrative 
resources 


Professional 

faculty/staff 


Provides a local- 
ly available pro- 
fessional and 
technical sound- 
ing board 


Provides sources 
for relevant 
literature and 
relevant chunks 
of information 


Not applicable 


Administrative 

Officers/Staff 


Establishes 
policies related 
to professional 
requirements and 
practices 


Establishes 
policies and pro- 
cedures related 
to information 
gathering acti- 
vities including 
library, travel, 
teleohone 


Establishes 
policies related 
to resource al- 
location and 
clerical support 


• 

External Environment 


Research Project 
Sponsors 


Degree of guid- 
ance covering 
the boundaries 

of the proposed 
research project 


Provides informa- 
tion on problem 
areas of high 
interest and the 
identification of 
other scientists 
with common 
interests 


Provides fiscal 
support and 
establishes ad- 
ministrative re- 
quirements and 
guidelines con- 
cerning the 

SUDDOrt 


Journal Editors/ 
Referees, and 
Professional Person- 
lel who review and 
approve convention 
papers 


Provides basic 
publication 
standards cover- 
ing technical 
requirements 


Provides basic 
publication stan- 
dards covering 
supporting litera- 
ture requirements 


Provides basic 
standards cover- 
ing clerical and 
format require- 
ments of an 
article/ present- 
ation 


Professional 
Community of 
Scientists/Engineers 


Provides formal 
methodological 
and subject 
matter inputs 


Provides descri- 
ptions of com- 
pleted research 
studies and 
findings via for- 
mal and informal 
means 


!^ot applicable 



and managerial nature. In what ways these different constraints, 
requirements , and services influence system planning will be 
described in the next chapter. 
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CHAPTER HI 

TECHNICAL DESCRIPTION 



III. Technical Description 



A. Introduction 

The role of the preceding chapter was to identify functional 
types of information needed as inputs to system planning and develop- 
ment decisions. The role of this chapter is to identify basic types of 
system-related decisions and to discuss how functional and technical 
inputs are used to help resolve these decisions. 

The following summary statements on scope and orientation 
provide a context for discussing system-related decisions. 

. The task-oriented systems of concern are restricted to 
advanced computer-based, on-line, remote access, time-sharing 
systems. 

. The alternative hardware and software system concepts, 
configurations, and components considered as potential candidates 
for the task-oriented systems are within the technological state-of-the- 
art. 

. The task-oriented R-P System described in this report is 
composed of two so-called independent subsystems , a scientist sub- 
system, and a service subsystem. The full range of tasks and subtasks 
required of grant- supported research projects are to be performed by 
the scientist and service subsystems either separately or jointly. 

. The orientation adopted in this report is that the service 
subsystem should either separately or jointly with the scientist 
subsystem assume primary responsibility for as many research 
project subtasks as possible. It is recognized, however, that a 
number of constraints will prohibit the attainment of this idealized 
goal. These constraints include cost, technology, professional 



and political considerations , and the current inability to adequately 
identify and describe the inputs, processes, and outputs associated 
with some of the research project subtasks; particularly, those subtasks 
requirinq creative and interpretative skills and knowledge* 

The service subsystem should be capable of performing or 
contributing to the performance of a wide range of research project 
subtasks requiring capabilities in the informational, managerial, and 

scientific areas. 

The computer-based system of concern to this report is that 
which is either located on the university campus or accessible through 
remote input/output devices. In either case, the system-provided 
services are administratively supported by the university. 

This chapter introduces and discusses four basic types of system 
planning and development decisions . One of the basic decision require- 
ments is to achieve an optimum allocation of research project functions to 
either the scientist subsystem, the service subsystem, or to both sub- 
systems for joint accomplishment. A second decision requirement is to 
achieve an optimum selection of a system concept; the third involves 
the selection of an optimum configuration or configurations of types of 
hardware , personnel , and software for the chosen system concept; and 
fourth, the selection of optimum components, that is, specif ic hardware , 
personnel, and software for the identified optimum system configuration(s). 



In resolving these four basic decisions, the types of information 
identified in the preceding chapter will be used as inputs. The information 
inputs may be grouped into one of two categories— functional requirements 
and constraints. The functional requirements category includes (1) the 
identified research project tasks and subtasks and variations associated 
with the methodological paradigms, (2) the inferred set of skills and know- 
ledge or performed capabilities needed to perform the research project 
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tasks and subtasks, and (3) the standards or criteria used to evaluate 
the completed research project. The constraint types of inputs include 
(1) data on the attitudes and research-related activities of the scientists 
across all disciplines, and (2) data on the structural, operational, 
temporal, and fiscal constraints arising from the university environment. 

The remainder of this chapter is divided into two major parts. 

The first part describes a decision model used as a framework for 
processing the system planning and development decisions. This part, 
described under Section B, includes an identification of the elements 
which make up the model, the grouping of these elements under four 
basic types of decisions required during system planning and development, 
and a general description of how these elements function during the process 
of decision making. The second part describes in greater detail each of 
the four basic system planning and development decisions. Each of the 
four sections in the second part includes identification of the decision 
problem to be solved, the types of alternatives to be considered, the 
required inputs for each type of decision, the criteria for resolving the 
decision, the nature of the decision output, and the process involved 
in successively screening, evaluating, and eventually selecting one 
of the proposed alternatives . Sections C through F cover this second 
part. 

B. Decision Model 

The decision model presented in this section was developed to 
serve as an aid for identifying, structuring, and resolving major decisions 
associated with initially screening possible system alternatives, evaluating 
the remaining feasible system alternatives , and eventually selecting rin 
optimum system. The elements of the decision model are introduced and 
described using university-based, grant- supported research projects as 
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the assigned system task. Although the substantive parts of the decision 
model encompass grant- supported research projects, the basic framework 
should be considered as generally applicable to other tasks which are 
concerned with information and data processing. 

The decision model is responsive particularly to two of the 
trends noted in Chapter I of this report; (1) the growing numbers of 
alternative system concepts , configurations , and components from 
which to choose in planning a computer-based system and (2) the diversity 
of orientations or perspectives associated with this selection process, 
that is, the system designer/developer, the system operator/maintainer, 
the system buyer and the system user. In response to the two trends 
noted above, the form selected is a graphic construct of the decision 
process . A simplified version of a cost/benefit decision model is 
presented in Figure 10. This graphic illustration shows that the decision 
model is composed of three major parts: the input segment which consists 
of characteristics or properties of the system inputs and information on 
constraints and requirements associated with the system outputs; the 
decision process , which consists of both the alternatives to be con- 
sidered and the criteria for screening and evaluating these alternatives; 
and the output segment, which consists of selected and rejected 
alternatives . 

For each of the four basic decisions identified above, there are 
three steps (see Figure 11). These steps are arbitrarily labelled as the 
constraint step, the feasibility step, and the selection step. The rationale 
underlying the three steps is based on a strategy of eliminating proposed 
alternatives according to a procedure which minimizes the requirements for 
cost and effectiveness data. That is, data involving the performance and 
cost characteristics associated with each proposed alternative. The 
constraint step represents a go/no-go condition for the proposed 
alternatives (see Figure 12). Only those alternatives which are judged 
to be compatible with respect to the various established constraints 
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Figure 10 . A Simplified Illustration of the Cost/Benefit Decision Model 
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Figure 11, Scheme for Reducing Possible System Alternatives 
to the Selected Optimum Alternative 
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Figure 12. Relationship of Criterion Values. Weights^ 
and Rules to the Nature of the Decision 



Decision Nature 


Criterion Values , 
Weights , and Rules 


Screen to identify compatible 
alternatives 

(Step I) 


Assign go-no go values to 
constraints 


Evaluate to select feasible 
alternatives 

(Step II) 


Assign minimally acceptable 
and maximally allowable 
values to relevant cost/ 
benefit criteria 


Weight criteria to select the 
optimum alternative 

(Step III) 


Assign weights to criteria to 
reflect their relative importance 
and devise a rule for combining 
the weighted criteria 
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are considered as candidates for the next or feasibility step* These 
constraints can be political, professional, technological, organizational, 
environinental , or cost in nature. The goal is to separate as quickly 
and inexpensively as possible compatible from incompatible alternatives . 
The second step, labelled the feasibility step, serves as a screen for 
the remaining compatible alternatives . Those compatible alternatives 
which either do not meet established minimum performance levels or 



exceed maximum cost levels are eliminated from further consideration. 
The selection step - the final step - involves the weighting of both 
cost/benefit criteria with respect to relative importance and the 
application of a rule for combining the weighted criteria to arrive at 
an optimum alternative. As one progresses from the constraint step to 
the selection step, data required on cost and effectiveness of each 
alternative increases in both degree of specificity and in volume. 

For this reason, the procedure shown in Figure 11 is designed to 
minimize the cost and performance data requirements while at the same 
time insuring that all possible alternatives are treated as potential 
candidates for the task-oriented system. 



C . Allocation of Research Functions 

A decision that must be resolved during the very earliest period 
in system planning concerns the allocation of job functions . Which 
functions will be the primary responsibility of the new system and which 
functions will remain the primary responsibility of the existing system 
is the nature of the decision problem. Here, the decision task is one of 
determining which research project functions the service subsystem will 
perform separately or jointly with the scientist and which functions, if 
any, the scientist will continue to perform. As noted earlier, the 
orientation adopted in this report is to assign to the service subsystem 
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primary responsibility for performing or contributing to the performance 
of as many research functions as possible. 

The allocation decision differs from the other three basic 
system planning and development decisions in a number of ways . 

First . it is concerned with the scope and performance boundaries 
of the service subsystem rather than with the nature or structure of the 
service subsystem, as are the other three decisions. Second, the process 
of allocating research functions is achieved totally by use of go/no-go 
constraints rather than criteria which can assume a number of values 
along various cost and performance scales. It is possible to pictorially 
characterize the process as one in which the system planner starts by 
assuming that the service subsystem will perform all of the grant- 
supported research functions . Then various filters , constraints , are 
used - constraints such as university policies, scientist research- 
related activities, and computer state-of-the-art - such that the end 
result of the filtering process results in three groupings: those functions 
that are assigned to the service subsystem, those functions that will 
be jointly accomplished by the scientist and service subsystem, and 
finally, those functions that remain the responsibility of the scientist. 
Third, the outcome of the decision process does not lead to the rejection 
of some of the alternatives as with the other three decisions . The 
decision task is not concerned with accepting versus rejecting but 
with allocating responsibility. Although the allocation type of decision 
is different, to preserve a structural and functional commonality through- 
out, the format used as a guide for allocating research functions is 
similar to the one used for the other three classes of decisions. 

What are the research functions to be allocated? In the pre- 
ceding chapter three primary types of skill and knowledge requirements 
were identified: scientific , managerial , and informational . An analysis 



m-9 



















PH 






of the research project subtasks led to "factorization" of these three 
main types of skill and knowledge requirements into 16 research sub- 
functions . These research functions comprise the kinds of performance 
capabilities required to satisfactorily plan, perform, and document a j 

grant-supported research project. Figure 13 presents the results of 
this preliminary analysis. j 

Examination of Figure 13 shows that there are eight scientific 
categories, six managerial categories, and two informational categories. 

The scientific category is divided into creative and technical requirements . 
Creative requirements are further subdivided into inductive and deductive 
skills. The inductive category includes those research activities which 
emphasize the requirement for generalizing from a specific empirical 
data base to a more general theoretical base; while the deductive category 
emphasizes the skills associated with deducing or hypothesizing specific 
testable consequences or outcomes from a broader theoretical base . 

Subtasks involving hypothesis formulation and hypothesis delineation 
fall into the creative category as well as the problem of interpreting 
research findings. The technical category is divided into methodological 
capabilities, content knowledge, and technique skills. Methodological 
requirements are further subdivided into those labelled as selection, 
design, and analysis. Selection includes those required capabilities 
associated with selection of such elements of a research project as a 
study setting (laboratory versus field) , selection or development of 
criteria, selection of conditions, selection of experimental subjects, 
and selection of appropriate analyses to perform on the data. Design, 
on the other hand, deals with the procedural characteristics of a 
research project. The design category includes how and when certain 
events will be manipulated or controlled, or measured, or sampled, or 
the data analyzed . While selection is concerned primarily with the 
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identification of what will be investigated and where, design is con- 
cerned with how and when the investigation will be conducted and the 
findings processed. Subject matter content is subdivided into those 
items of required knowledge (stored information) which are specifically 
problem-related versus those required items of knowledge which are 
more generally discipline-related. Finally there are subtasks which 
emphasize manipulative skills and knowledge versus those requiring 
measurement skills. 

The managerial category is divided into clerical and resource 
management requirements . Clerical categories include information 
handling such as filing, organizing, coding, and compiling; information 
generation such as typing, writing, displaying, etc.; and quality control 
such as detecting errors of omission and commission, and editing. The 
resource management category includes (1) the allocation of resources; 
that is, personnel, facilities, equipment, and money to accomplish 
different parts of the research project, (2) the coordination or utilization 
of the resources over time through scheduling, and (3) the provision of 
necessary guidance or instructions to participating research staff 
members to insure efficient conduct of the research project. 

The informational category pertains to those subtasks which 
involve the acquisition of information from external sources and the 
processing of this information. This includes all subtasks requiring 
bibliographic types of activities , the reviewing , screening and ex- 
tracting/tagging of relevant chunks of information, and analysis of 
these "chunks of information" . The informational category is divided 
into the collection/organization subcategory and the analyst s/inter- 
pretation subcategory. The former includes all subtasks which involve 
the more mechanical aspects of information processing while the latter 
refers to the more intellectual and analytical activities associated with 
information processing. 



Figiire 14 provides a framework for grouping subtasks by skill 
and knowledge categories for each phase of the research project and for 
each of the four methodological paradigms . It can be seen in the figure 
that different skill and knowledge categories are employed in the detailed 
planning phase depending on the methodological paradigm selected . 

Figure 14 may be used also to point out that as the system planning 
and development activities progress, the size of the data base grows 
exponentially. Therefore, it becomes increasingly important to classify 
and organize system-related information and data. The system data base 
is used as inputs to decision making and as a source of information to 
facilitate the development of alternative system concepts (64,79) . 

Figure 15 illustrates that three successive screening steps or 
types of constraints are used to guide the allocation process. The 
initial screening step utilizes functional types of constraints to help 
determine which research functions will be potentially the primary 
responsibility of the service subsystem, the joint responsibility of 
the scientist and service subsystem, or remain the responsibility of 
the scientist. The major functional constraint concerns present limita- 
tions on being able to adequately define all of the required tasks and 
subtasks associated with grant- supported research projects^ Those 
tasks and subtasks which cannot be adequately identified and described 
must necessarily remain, for the time being, the primary responsibility 
of the scientist. A secondary constraint involves professional and/or 
political views held by the population of scientists in question. An 
overwhelming majority of the scientists may, for professional or political 
reasons , insist that the performance of certain research subtasks remain 
their primary responsibility. However, it is recognized that attitudes 
of Individuals, including scientists, are subject to change with in- 
creasing opportunity to share in the benefits of modem technology. 
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Detailed Planning 
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Figure 15 . The Allocation of Research Project Functions 
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For this reason, and because it is consistent with the task-oriented 
approach, less weight is placed on attitudes as a system constraint 
’ than on the ability to articulate research tasks and subtasks . The 
second filtering step utilizes what might be conveniently labelled 
as 'university' constraints. Specifically, the problem is to separate 
research functions which are unigue to particular paradigms . These 
unique research functions will be retained by the scientist and support 
staff (e.g. , a secretary) for primary accomplishment. The obvious 
purpose of this filtering step is to retain only those functions for 
further system planning activities which are judged to be worth spending 
time and money. That is, functions which are common to a wide range 
of academic scientists . The final screen on the remaining functions is a 
technological one . The general state-of-the-art in computer technology 
serves as a constraint. In the context, the phrase, "state-of-the-art" 
is taken to mean "off-the-shelf" variety rather than theoretically possible 
or even research-demonstrated state-of-the-art capable. It is recognized 
that special computer software programs will need to be developed and 
that items of hardware will need to be combined in perhaps unique ways . 
However, these fabrication types of activities do not require research 
and development (R&D) effort in the commonly understood sense of R&D. 
During this filtering step, the question is whether the service subsystem 
can assume primary responsibility for accomplishment of the remaining 
functions or whether - due to current limitations in computer technology - 
the scientist must continue to perform certain parts of the research 
functions and the service subsystem the remaining parts . 



D. Selection of a System Concept 

The goal of this system planning activity is to select from 
among proposed alternatives a concept which is optimum with respect 
to managerially oriented cost/benefit criteria . 
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Considering the wide range of scientific, informational, 
and managerial capabUities needed to perform the tasks and subtasks 
which make up a grant-supported research project , it is likely that 
proposed altenative system concepts will differ significantly among 
each other In terms of both their comprehensiveness in performing the 
allocated research functions and In the manner in which these functions 
will be performed . VariabUity and multiplicity in system concepts is critical 
if the planning process, as presented in this report, is to possess any 
utility. The larger the number of innovations proposed, the more likely 
there will be system concepts which differ radically in their character- 
istics . Trade-offs will be required between numbers and kinds of 
research functions to be included within the various system concepts . 

For example, one system concept might maximize on those research 
functions of an informational nature in contrast to the scientific or, 
managerial functions. As another example, primary attention might be 
devoted to a system concept which emphasizes performance of research 
functions which have been allocated to the scientist and service sub- 
system for joint accomplishment rather than those functions aUocated 
to the service subsystem for primary responsibility. The point to be 
made is that the diversity and range of functional requirements are 
sufficiently broad so that it is most probab hat alternative system 
concepts, which are proposed, will vary considerably in their degree 
of comprehensiveness and emphasis, and in the form they will assume. 
These variations in alternative system concepts provide the bases by 
which concepts are compared in terms of various managerial criteria . 

Figure 16 presents three managerlally-oriented criterion concepts 
and possible operational definitions for these concepts. As the figure 
indicates, the primary perspective is that of the system buyer. Indivi- 
duals, institutions, or organizations which contribute money, time. 



talent, and/or facilities represent system buyers. In this context 
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Figure 16 . Criterion Concepts an d Measures 
Which Reflect the Managerial Point of Vlevg: 
















Ill- 18 







growth potential is measured in terms of the modifiability or adapt- 
ability of a system with respect to its structural, input/output, and 
performance characteristics . Since the state-of-the-art in projecting 
future requirements is debatable regarding its accuracy, a strategy 
used in system planning has been to design and develop system 
concepts which are modular or evolutionary in nature . That is , 
systems are designed to facilitate modifications in their structural, 
input/output, and performance characteristics. In addition, the trend 
has been to shift more of the functional requirements from the less 
modifiable hardware to the more modifiable software or computer 
programs . The criterion concept of marketability cited in Figure 
16 refers to the ability of a system to perform either at the same 
time or within a given period of time a number of functions . The 
extent to which the system is available for use by different scientists 
at the same time increases the feasibility of having more scientists 
use the system. Remotely accessible, time-sharing system capabilities 
enhance the marketability of a system in that they permit a number of 
scientists at different locations to simultaneously perform research 
activities jointly with the computer. The criterion concept of flexibility 
pertains to the number of different research project functions that the 
computer-based system can perform; that is, informational, scientific, 
and managerial functions. 

System concepts are not only screened and evaluated with respect 
to the management oriented criterion concepts - Growth Potential , 
Marketability, and Flexibility - but also in terms of projected costs. 

The total costs incurred in both acquiring and operating a system are 
included. Theoretically, each proposed system concept can be re- 
presented in terms of projected values on each of the three effective- 
ness dimensions and on each of the two cost dimensions . A hypothetical 
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illustration of three alternative system concepts is presented in 
Figure 17 (96). The extent to which each alternative varies along the 
effectiveness and cost dimensions is graphically represented by its 
distance along each of the vectors . The further out the line intersects 
the three effectiveness vectors, the higher is the value associated with 
a particular alternative. In contrast, the further out the line intersects 
the two cost vectors, the lower is the cost value associated with parti- 
cular alternatives. By defining cost/effectiveness relationships in this 
manner, the further out a line intersects a dimension, the more 
desirable is the outcome. Two important points are illustrated in this 
figure. First , system criteria which reflect multiple performance 
reguirements and cost constraints are often conflicting. For 
example. System Concept C, shown in Figure 17, is best with respect 
to acquisition costs. And, System Concept B is highest in Marketability 
but poorest in Flexibility. Obviously, not all criteria conflict and 
the existing technological state-of-the-art contributes a great deal 
to whether and how much various criteria conflict. As an example, 
for years high information storage capacity in a computer was likely 
to be attained at the expense of less rapid information processing 
rate. Today this conflict is less evident (except at extreme speeds 
and capacities) because of advancements made in computer technology. 
Second, the fact that proposed system concepts often differ among 
themselves with respect to values ac.iievable on different criterion 
dimensions makes the problem of criterion weighting an important 
one. Each criterion dimension must be weighted as to its relative 
importance as a guide in helping to select an optimum alternative (100). 

The process of screening, evaluating, and eventually selecting 
an optimum system concept is portrayed in Figure 18. The first step 
is to identify and screen out alternative system concepts which are 
incompatible with either structural properties and operational policies 
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Figure 18. Selection of a System Concept 
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of the university environment in which the system will be located 
(e.g. , building size and air conditioning capabilities) or with the 
research related behavioral and attitudinal characteristics of the 
academic scientists. The second step is concerned with screening 
out remaining alternatives which are found to be infeasible in terms 
of established cost/effectiveness criteria. The final step involves 
selecting one alternative which best meets the combined and weighted 
cost/effectiveness criteria. Obviously, as one alters the criterion 



values , weights , or rules for combining the weighted criteria , the 
outcome changes . Making explicit the bases or reasons for these 
values, weights, and rules is an essential part of the system 
planning and development process (103). 



As an aid in conveying the scope and nature of a system concept. 
Figure 19 was prepared. This figure presents a simplified graphic version 
of a hypothetical system concept. There are a number of salient points 
which should be stressed. These points are briefly described below. 



. The system concept recognizes subject matter differences 
and methodological differences among the physical , biological , and 
social sciences. In the example shown, these differences led to a 
three-part structure among the university-based academic researchers . 
As shown in the figure, a system concept may consist of several 
structural/functional units . This involves the grouping of functional 
requirements within various existing organizational and/or structural 
parts of the university context or whatever platform or context in 
which a system is being overlaid. In the example shown, there are 
five structural/functional units . These are identified and described 
next. 

. The portrayed system concept reveals (l) an "allocation" 
of the required research functions among the different organizational/ 
structural units and (2) a notion of how these distributed research 
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functions will be performed. For convenience, these five different 
types of structural/functional units are labelled as system configurations. 
The five configurations include the computer-based service subsystem (A) , 
the joint scientist-service subsystems (B), the commercially leased 
services (C) , the interdisciplinary teleconferencing network configura- 
tion (D) , and the discipline-based scientist configuration (E) , 

, Because the illustrated system concept is made up of five 
different configurations, the system planner must not only be con- 
cerned with the hardware/s oftware/personnel make-up of each con- 
figxaration but also with the input/output interfaces which functionally 
and structurally tie the five configurations together into a complete 
R-P System, 

, The graphic system concept furthermore shows that the five 
configurations have been assigned varying amounts and different types 
of responsibility for performing the research functions. For example, the 
service subsystem configuration (A) has a major responsibility for per- 
forming several of the managerial and informational functions. Next, the 
joint scientist/service configuration (B) has been assigned a number of 
the scientific tasks. The commercial configuration (C) has been assigned 
responsibility for performing some of the informational functions involving 
specialized subject matter; the interdisciplinary teleconferencing network 
configuration (D) is designed to support research projects of a distinctly 
interdisciplinary nature; and the discipline based scientist-subsystem 
(E) has been assigned methodological tasks and subtasks which are 
peculiar to each discipline. 

E . Selection of System Configurations 

The problems of screening and evaluating system configurations 
introduces the notion of "process" criteria. Process criteria deal with 



the functional characteristics of a system. The objective of the decision 
maker is to select those configurations which (1) most efficiently 
(performance/cost) perform the various assigned subtasks and (2) are 
compatible with the Inputs and outputs of other configurations which 
make up the R-P System. Performance of the system configurations is 
measured in terms of accuracy, speed, and completeness vrith which 
each subtask, task, phase, and, ultimately the overall research 
project is Implemented. The operational definition of each criterion 
measure is a function of the specific characteristics and performance 
requirements associated with each subtask and task, and the structural 
and functional properties of the system selected. That is, the type and 
level of performance needed to satisfactorily complete each part of the 
research project and the nature of the system dictate both the type of 
the measure to be used (accuracy in what terms , completeness in what 
terms , and/or time constraints in what terms) , and the level of perform- 
ance required (how accurate, how complete, and/or how fast) (9, 46) . 

Acquisition and operational costs are also involved in helping 
to screen and evaluate proposed configurations , Estimated costs for 
proposed system configurations should be more specific and reliable 
than estimates made for system concepts. However, these estimates 
will still not approach the exactness of component costs and effective- 
ness estimates simply because, when the selection of components is 
made, components are being considered rather than a tyE£ 

of component (a certain type of person versus a certain type of hard- 
ware), The "individual" most interested in meeting the process criteria 
is the system operator (i.e. , the scientist subsystem and service 
subsystem) . 

Figure 20 was prepared as an aid in communicating the nature 
of system configurations . This figure illustrates in a very simplified 
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manner an outcome of allocating specific task activities to hardware, 
software, and personnel components. The example shown in the figure 
identifies which resource management activities have been assigned to 
what types of components within a given configuration. The figure 
presents not only the allocation“Of“tasks~tO“Components but also 
identifies the order in which the activities of each task will be 
performed and what types of outputs will be transferred from one 
type of component to another. Performance requirements for each 
task and its associated activities coupled with the performance 
capabilities and limitations of each proposed type of component 
serve as inputs to the evaluation process. 

The process of selecting system configurations is shown in 
Figure 21. Using the example given earlier, there are four structural/ 
functional units for which the system planner is interested in configuring, 
that is, selecting optimum combinations of hardware/software/personnel . 
The fifth structural/functional unit is the scientist subsystem who functions 
as a given in the system design. 

The first step in this system planning process is concerned 
with identifying configurations which have internally compatible input/ 
output characteristics. That is, proposed configurations for each of 
the different structural/functional units is made up of some mix of 
hardware/software/personnel components described at a type or 
broad capability level rather than in terms of specific brand names, 
individuals, or specific design capabilities. An important require- 
ment is to initially screen out possible mixes or configurations of 
hardware/software/personnel whose input/output interfaces internal 
to the configuration are not compatible with respect to the required 
performance characteristics. Technological (for hardware and software) 
and behavioral (for human) constraints are used to screen out in- 
compatible configurations . These constraints may be structural , 
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functional, or operational in nature. This compatibility exercise is 
required for each of the different structural/functibnal units which are 
part of the system concept. In the example shown, the compatibility 
exercise is required for the four structural/functional units. 

The next step is to identify those configurations which meet 
established performance and cost criteria. Again, the step must be 
done for all of the system's different structural/functional units. 

The final step involves the selection of optimum configurations; 
one for each of the different structural/functional units . Weighted cost 
and performance criteria are combined according to a particular rule and 
applied to each of the applicants . It should be noted that both the 
performance and cost criteria will likely be different for each of the 
structural/functional units of the system. Likewise the criterion 
weighting scheme may change. In some cases the rule for combining 
the weighted criteria may be altered depending on the criticality and 
make up of the structural/functional unit in question. The end result 
of this decision process is a set of optimum configurations of types 
of hardware/software/personnel components. 

F . Selection of System Components 

At this descriptive level in system planning, the number of 
times that the decision process must be exercised proliferates 
tremendously. For each configuration, there are generally a number 
of different types of components. Also, between configurations, there 
are a number of interfacing input/output components to be selected . 

It is most critical to screen out, as quickly as possible those 
components which are found to be incompatible. It will be necessary 
to gather comprehensive cost effectiveness data as inputs for the 



III-30 









subsequent evaluation process for components which survive the 
initial screening . One technique would be to treat initially those 
specific components which will contribute to the performance of a 
number of research activities . It is likely that a number of similar 

i 

or identical components will l|)e used not only in the same configuration 

i 

but also in different configurations. If this is the case, much of the 
available cost/performance data on these components will be applicable 
more than once in the evaluation process. 

The evaluation process is achieved by the use of component 
cost and "reflexive" criteria. Figure 22 presents alternative reflexive 
criterion concepts and possible criterion measures applicable to the 
component selection process. The development of so-called component 
or reflexive criteria represents a rather complex distillation of both 
product criteria and system criteria . 

Figure 23 was prepared to graphically summarize the logical 
analyses required to derive the reflexive criteria. As shown in the 
figure, the nature and scope of the reflexive criteria are shaped by 
both the process and managerial criteria. It also shows that the process 
criteria are derivable from an analysis of the intermediate criteria. The 
criterion development task is probably the most critical, the most complex, 
and the less attended to problem in the field of system planning. A 
report prepared earlier on this project (102) structures and discusses a 
number of aspects associated with the criterion development problem. 

Figure 24 provides a schematic representation of the decision 
process associated with the screening, evaluating, and eventual 
selection of system components . Two points are made about this 
decision process. First, the initial screening process is concerned 
with identifying components whose design properties are compatible. 

The question of compatibility in selecting configurations is concerned 
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Figure 22, - Some Possible Reflexive Criterion Concepts and Measures 
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Figure 23. Criterion Model; A Summary 
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A. ULTIMATE CRITERIA (REFERENT IS A COMPLETED STUDY OR REPORT) 

1 . Adds to existing knowledge = adequacy 

2 . Active interest in study/report = pervasiveness 

3 . Wide application or generality = survivability 

B. INTERMEDIATE CRITERIA (REFERENT = COMPLETED TASKS AND SUBTASKS) 

1 . Quality = qualitative and quantitative concepts as accuracy, 
goodness, and worth 

2. Timeliness = speed, duration, and reaction time 

3. Nuinerosity = frequency , size, and completeness 

C . MANAGEMENT CRITERIA (SYSTEM CONCEPT IS THE REFERENT) 

1 . Growth Potential - Modifiability of system 

2 . Marketability = Number of subtasks processed in X time 

3 . Flexibility = Percent of total research project \ 

D. PROCESS CRITERIA (SYSTEM PERFORMANCE IS THE REFERENT) 

(Quality, Timeliness, and Numerosity) 

E. REFLEXIVE CRITERIA (SYSTEM COMPONENTS ARE THE REFERENT) 

(Performance, Availability, Utilization) 
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Figure 24. Selection of System Components 
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with the general performance capabilities and limitations of various 
types of components. Here, we are concerned with the specific 
design features of particular components. Again, the question of 
internal compatibility (within a configuration) and external compati- 
bility (across configurations and external to the system itself) must 
be resolved. Second, the magnitude of the task of screening, 
evaluating , and selecting components would probably tax the strength 
of even the most dedicated system planner. However, there are a 
number of aids or procedures possible. Some of these aids and pro- 
cedures include: (1) identifying points of sensitivity for the various 
components, that is, design properties most susceptible to the re- 
quired range in performance thus permitting a component to be rejected 
on the basis of a single factor, (2) identifying "built-in" dependencies 
across components, that is, interactive characteristics among func- 
tionally or structurally related components , and (3) concurrent evaluation 
of several components which may allow the planner to screen out a 
number of candidates at one time. Although there are a number of aids 
or procedures which may facilitate fast screening and evaluation, the 
basic proposition underlying the system planning guide is that all 
reasonable alternatives will be considered. 

Prior to concluding this chapter, it may be useful to point out 
that the process of successively delineating the system characteristics 
is not completed when the particular system components have been 
selected . The selection of particular components now permits the system 
planner to describe in greater detail the specific actions to be taken by 
each component at each phase in planning, conducting, and documenting 
the research project. The design properties of the components selected 
influence the particular sequence and nature of activities that will be 
performed. The resulting detailed description of system operations 
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provides the basis for preparing operating and maintaining documents 



and materials for instructing the scientists and service subsystem 
personnel in the operation of the various system components • 



\ 









III-36 




















_ — — 



CHAPTER IV 

ADMINISTRATIVE DESCRIPTION 



IV. Administrative Description 



A. Introduction 

Two of the three major sets of activities associated with 
system planning and development have been covered in Chapters II 
and III. This chapter covers the third set of activities — management 
of available resources including time/ money, facilities, and personnel. 

During the initial phase of this project, from March 1967 to 
December 1967, primary reliance was placed on utilizing available 
written sources of information as inputs for the functional and techni- 
cal aspects of system planning. In the current phase of the project, 
in addition to utilizing written materials, relevant data and information 
were acquired through on-site observations and interviews with 
information system planners and managers at a number of universities . 
Conferences were arranged with individuals actively engaged either in 
developing and managing specialized information systems or in planning 
the modernization of large university libraries . Concurrent with this 
shift to on-site observations and interviews, it becam^ apparent that 
the managerial requirements associated with planning ^nd developing 
complex specialized information systems and library modernization 
programs are most critical. The inherent complexity and demands of 
the managerial task are well known to those who have been actively 
involved in the evolution of either military or civilian computer-based 
information systems . However, surprisingly little relevant documentation 
on system management is available to serve as a guide for those who 
have had little or no experience in managing the planning, development, 
and operation of complex information systems. A possible explanation 
for the sparsity of documented experiences concerns the natural and 
understandable hesitancy to put into writing problems experienced. 
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as well as findings concerning successful and unsuccessful solutions 
found to these problems . 

The observation concerning the criticality of the managerial 
role in system planning and operation was further reinforced during 
the 1968 annual meeting of the American Society of Information Sciences 
(ASIS) in Columbus , Ohio. Although the formal presentations and papers 
had virtually nothing to say about the managerial aspects, informal 
discussions provided substantial information and commentary on various 
aspects of system management. On the basis of these two sets of 
experiences, the decision was made to place greater emphasis on the 
administrative or management aspects of system planning and development 
than originally had been planned . 

Because of the already mentioned sparsity of relevant documentation, 
it was necessary to rely on three rather diverse sources of information i 
(l) system management -related information and suggestions offered by 
individuals during visits to selected universities and during the annual 
ASIS meeting, (2) documents discussing management aspects associated 
with system planning in general , and information systems planning in 
particular, and (3) personal experience with U.S. Naval planners and 
scientists during earlier research projects on computer-based command 
control systems . 



The administrative or managerial aspects of system planning and 
development covered in this chapter are organized into three parts: (1) defini- 
tions of basic resource and information management functions, (2) identifi- 
cation of tasks involved in establishing a management organization, and 
(3) a comparison of management requirements during the developmental 
versus the operational phases of a system. 
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B. System Management Functions 



Figure 25 lists and broadly defines basic resource and informa- 
tion management functions. In this context, the resources referred 
to are those which have been allocated to the planning and development 



elude money, time, facilities, and personnel assigned to support the 
programmatic evolution of a system. The types of information required 
include not only information about the system planning and development 
resources, but also information about the resources which will logistically 
support the operational system. In other words, information requirements 
are of two sorts: information about resources used in supporting the plan- 
ning and development of the system and information about resources 
needed to support the system when it becomes operational . The major 
responsibility of the system planner, functioning in the role of a 
manager, is to effectively match available resources or assets against 
system planning and development needs . To effectively accomplish this 
responsibility the system planner must have information about (1) the 
resources in terms of their availability, capabilities, limitations, 
locations, and numbers, (2) the system planning and development 
needs in terms of their functional and technical properties, temporal 
or sequential characteristics, and magnitude, and (3) the operational 
context in which the resources and needs will be mcitched (particularly 
those aspects of the context or environment which may significantly 
influence either the needs, the resources, or the interaction of resources 
and needs) . One way of conceiving the management role is that the 



^ The situation frequently exists during the life cycle of an information 
system in which the same resources are used to support system planning 
and development as are used to support a system when it is operational. 
The potential confusion resulting from this dual function of resources is 
brought up again at the end of this section. 






of the system as distinct from the system itself.^ The resources in- 
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Figure 25 « Resource and Information Management Functions 



Resource Management Functions 

Allocation ; Assigning and reassigning (as necessary) available 
resources (money, personnel, facilities, and time) to accomplish 
required system planning and development tasks . 

Coordination : Establishing communication procedures and time 
schedules to insure efficient utilization of available resources . 

Guidance : Preparing and disseminating Instructions, policy 
guidance, and training materials to insure effective and 
directive application of resources . 



Information Management Functions 

Acgulsition; Includes subfunctions such as requesting, collecting, 
retrieving , and extracting information and data . 

Analysis ; Involves the evaluation of acquired data and informa- 
tion with respect to its relevance, completeness, and timeliness. 

Process ; Includes subfunctions such as compiling, formatting, 
indexing, disseminating, and storing relevant information and 
data . 
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system planner manages resources via management of information (96). 

The management of resources via management of information is performed 
in two different modes; i.e. , a planning mode and a monitoring mode. 

The planning mode involves the allocation of resources through time to 
meet the projected system and development needs while the monitoring 
mode involves the supervision or checking of the actual utilization 
against the planned utilization of assigned resources. The planning mode 
matches resources to needs through a projected future time period while 
the monitoring mode compares present activities against projected activities. 

The following discussion provides two examples of requirements 
associated with the system management functions . The first example 
deals with a resource management requirement; the second example is 
concerned with an information management requirement . 

. One resource management characteristic common to complex 
information systems is the dual role assigned to some of the resources. 

That is, the same facilities, equipment, personnel, and money are 
required to support both the planning and development of a new system 
and the operation of a current system. The overlapping of developmental 
and operational phases during the programmed evolution of computer-based 
information systems tends to blur the already arbitrarily defined system 
phases (82). While the system is in an operational mode, and decisions 
are being made and actions are being taken to modify, replace, or add 
on to the present system, many of the same resources are used to support 
the present and future systems. This dual role can quite easily produce 
confusion leading to a situation in which either or both of the system 
activities (developmental or operational) are degraded. Gotterer (33) 
in his article on Organizational Philosophy and the Computer Center 
proposes that two distinct suborganizational units be established. 

According to Gotterer, one suborganizational unit within the computer 
center should be assigned responsibility for the operation of the current 



IV-5 



system and one suborganizational unit should be assigned responsibility 
for the new or improved system. Gotterer further states that the two 
suborganizations should be composed of individuals possessing different 
qualifications, interests, and traits. Personnel requirements associated 
with the developmental phases versus the operational phase of a computer- 
based information system are compared in the final section of this chapter. 

. One information management characteristic associated with 
the long term evolution of complex information systems is the critical 
need for complete and accurate documentation of the decisions made 
during system planning and development. There is evidence to indicate 
that this critical need is not typically met (96). Some of the reasons 
for maintaining a complete record of system planning decisions , bases 
for decisions , and outcomes are fairly obvious . First, this type of 
information serves as required inputs when the system is evaluated 
at some later period • Second , future system planning and design can 
be greatly facilitated by these records. Third , the system planning 
and development state-of-the-art can be effectively advanced by build- 
ing onto and synthesizing these documented experiences . This critical 
need for careful system documentation requires that a special information 
storage and retrieval system be designed and used during the system 
planning and development cycle and transferred to those who, in the 
future, will be given the responsibility for the next generation system. 

C . System Management Tasks 

System management components include personnel and equipment 
which contribute to the performance of resource and information manage- 
ment functions , the organizational structure which reflects an allocation 
of management functions to different combinations of personnel and 
equipment components , and the communication and operational procedures 



which effectively link all of the components together within the 
organizational structure . In establishing a system management 
organization, there are at least five major tasks that a system 
manager or, in this context, a principal investigator (P J.) needs 
to perform. These five tasks are covered in the following paragraphs. 

An initial task required of the principal investigator in setting 
up a management organization is to define the types of information and 
resources needed to support the planning and development of a computer- 
based information system. Using the R-P System as an example, the 
information and resource requirements may be grouped into one of 
three categories; functional, technical, and administrative. Functional 
information and resource requirements encompass the full set of 
descriptive items covered in Chapter II, and the necessary resources 
needed to collect, process, and utilize this set of information items 
during the system planning and development phases Technical informa- 
tion and resource requirements include data relevant to computer-related 
hardware, software, and personnel items, and the necessary technical 
resources needed to plan, develop, and test system concepts , configura- 
tions , and components . Administrative requirements include clerical , 
managerial , and logistical support needed to effectively perform the 
resource and information management functions during both planning 
and monitoring activities . Information and resource support are needed 
to perform the resource allocation, coordination, and guidance functions; 
and, the acquisition, analysis, and information processing functions 
required of system management . 

A second system management task is to identify sources for 
obtaining the required information and resources. The most immediate 
source is the staff and material assigned to the system planning and 
development project. The university environment provides a second 
major source of system information and resources. The computer center 
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located in the university context is a rich source of technical support . 

The university faculty can provide functional or research-related items 
of information and resources and the university administration is the 
logical choice for administrative support . Sources, external to the 
university, may be local, regional, or national in character. Professional 
societies and research organizations are two examples of sources for 
functional or research-related information. Computer manufacturers 
and companies which commercially lease computer services - both 
hardware and software - represent two obvious sources for technical 
support. Finally, individuals and organizations knowledgeable in the 
areas of information system planning and resource management offer 
two alternative sources for meeting administrative information and 
resource requirements . 

A third system management task involves the allocation of 
information and resource management functions to the assigned system 
planning and development staff. The system planning components 
of interest to this task include both the personnel and supporting 
equipment and materials assigned to the project. Qne possible procedure 
is to base the assignment on the nature of the information and resources 
required; that is, functional, technical, and administrative . According 
to this scheme, there would be designated a functional or research 
director, a technical director, and an administrative director. All of 
the directors would be responsive to the principal investigator. A 
second possible procedure is to allocate responsibility based on the 
functions involved; that is, resource and information management 
functions. One member of the staff might be responsible for one or 
more of the information functions - acquisition, analysis, and process- 
ing - and another member responsible for one or more of the resource 
management functions; that is, resource allocation, coordination, and 
guidance . 
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A fourth system management task requires the development of an 
organizational structure . The organizational structure reflects the above 
allocation of management functions to different combinations of personnel 
and equipment components. Figure 26 was drawn to illustrate different 
organizational structures. The vertical structure possesses a greater 
number of intermediate links . Potential advantages gained with the 
vertical structure include the probability of obtaining more complete 
information at a relatively low level of cost at each echelon. Potential 
disadvantages of the vertical structure include a higher probability of 
information distortion and slower responsiveness in meeting rapidly 
changing information needs. By contrast, the potential advantages of 
the horizontal structure include faster response time in meeting 
information and resource requirements ~ due to the fewer number of 
links ~ and higher reliability of the data obtained from the various 
sources . 





The fifth and final system management planning task involves 
the formulation of communication and operational procedures which 
effectively link all of the system components together within the 
organizational structure. The established communication and opera- 
tional procedures will include assignment of responsibility for 
administrative decision making, procedures for resolving information 
and resource management types of decisions , information and data 
needed as inputs to these decisions, administrative criteria for helping 
to resolve these decisions, procedures for collecting and processing 
the needed decision input data and information, lines of communication 
connecting the various components of the management organization, 
coordination policies and practices , and procedures for determining 
and reporting system planning and development status and updated plans . 
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Figure 26. Comparison of a Horizontal 
vs . a Vertical Organizational Structure 
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D. Developmental vs. Operational Phases 

During visits to the various universities and at the annual 
ASIS meeting, a topic of common interest concerned management 
requirements during the system planning and development phases 
versus those required with an operational system. Based on (1) a 
range of experiences gained by those interviewed and (2) the writings 
of individuals who have specialized in management of systems during 
both the system development and operational phases, Figure 27 was 
prepared (8, 29, 45, 60, 70, 71, 82, 101). This figure lists some 
of the managerial requirements for the development and operational phases. 
The list is organized with respect to seven categories: primary orientation 
or objective, operating doctrine, organizational characteristics, re- 
source management characteristics, information management character- 
istics, management/staff characteristics, and logistical characteristics. 
Obviously, this list reflects individual experiences with particular 
systems and may or may not be applicable to other than the particular 
situation from which they were gained. However, there seems to be 
a degree of intuitive reasonableness about the list sufficient to consider 
this as a point of departure for accumulating, under more systematic 
circumstances , data on managerial requirements covering a wide range 
of computer-based Information systems . 

The information contained in Figure 27 does not imply that 
different Individuals will be required to manage the developmental and 
operational phases. Rather, the figure suggests that behavioral and 
operational goals and requirements are different. It is conceivable but 
unlikely that the same individual will possess the full set of qualifications 
and Interests to satisfactorily perform the management role under both sets 
of circumstances . 
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Figure 27, Management Characteristics 
as a Function of System Status 



Status 

Categories ^ — > — 


Develoomental 


Operational 


Primary Objective 


Problem solving (specific 
to basic Duroose of svstem) 


System viability (growth in 
number of users and services) 


Operating Character- 
istics 


1 . Mid and long range plan- 
ning 

2 . Changes reactive to situ- 
ational needs 


1 . Monitoring ongoing opera- 
tions 

2 . Programmed changes to 
meet planned objectives 


Organizational 

Characteristics 


1 . Organized around skill/ 
knowledge requirements 
(subject matter, technical, 
administrative 

2. Centralized decision 
making 

3 . Maximize informal lines 
of communication/coordina- 
tion 

4. Organizational changes 
are a function of shifting 
requirements (subject matter 
to technical to administrative) 


1 . Organized around system 
functions (document acqui- 
sition, surrogation, etc.) 

2 . Decentralized decision 
making 

3 . Maximize formal lines of 
communication/coordination 

4. Organizational changes 
are a function of growth in 
size and coverage 


Resource Management 
Characteristics 


1 . Personnel/equipment re- 
sources perform multiple 
roles 

2 . Decentralized control of 
resources 

3 . Maximize informal and | 

inter-personal forms of co- 
ordination 

4 . Maximum use of broad 
and verbally delivered 
guidelines 

5 . Maximize re-allocation 
of resources to meet specific 
needs 


1 . Personnel/equipment re- 
sources perform single roles 

2 . Centralized control of re- 
sources 

\? . Maximize formal and 
written forms of coordination 

4 . Maximum use of specific 
and written forms of guidance 

5 . Minimize the practice of 
reassigning resources 


Information Management 
Characteristics 


1 . Centralized control of re- 
source status and system 
progress information 

2 . Maximize acquisition of 
information on nationwide, 
individual and organizational 
skill s/knowledge , available 
services, and off the shelf 
items of hardware and soft- 
ware 


1 . Decentralized control of 
resource status and system 
operating information 

2 . Maximize acquisition of 
information on locally avail- 
able potential users of system 
services, sources for support- 
ing new services, and tech- 
nological state-of-the-art 
growth 
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Figure 27 (Cont.) 



Status 

Cateaorles 


Develoomental 


Operational 


Information Management 
Characteristics (cont.) 


3. Emphasize documentation 
of system planning decision 
and bases for decisions 

4. Data used as inputs to 
decision making are often 
incomplete, general, and 
unreliable 

5. System-related informa- 
tion tends to deteriorate 
rapidly in its utility 


3 . Emphasize documentation 
of system useage 

4. Data used as inputs to 
decision-making tend to be 
more complete, specific, 
and reliable 

5 . System-related informa- 
tion tends to deteriorate 
slowly in its utility 


Management/Staff 

Characteristics 


1 . Creatiye abilities 

2 . Flexible in behayior 

3 . High inter-personal skill 

4 . Problem oriented 

5. Subject matter skills 

6 . Capability oriented 

7. Performance sensitiye 


1 . Executiye abilities 

2 . Organized in behayior 

3. High product -related skill 

4 . Organizationally oriented 

5. Managerial skills 

6. Function oriented 

7. Cost sensitiye 


Logistical Character- 
istics 


1 . Must deal with unantici- 
pated acquisition costs 

2 . Frequent non-scheduled 
requirements for logistical 
support 

3 . Deyelopmental and make 
shift items 

4. Emphasis on fabrication 
and testina 


1 . Usually deals with known 
operating costs 

2 . Scheduled requirements 
for logistical support 

3 . Standard and off-the- 
shelf items 

4. Emphasis on maintenance 
and operation 



The primary objective and operating characteristics shown in 
Figure 27 are the basic factors which dictate the nature of the other 
management characteristics . During the system planning and development 
phases, major emphasis is given to problem solving types of activity/ that is, 
generating alternative solutions and selecting optimum solutions which 
meet established functional, technical, and administrative requirements. 

In contrast, when the system is operational, major activities shift from 
problem solving to those designed to maintain the viability of the system. 
Furthermore, management, during the planning and developmental phases 
of a system, focuses on mid- and long- range planning for two reasons. 

First, mid- and long- range planning offer a means of providing continuous 
direction and guidance to system planning and development activities . 

Second, unforeseen obstacles and problems which occur in the present 
time period require management to periodically update the mid- and long- 
range plans. For these reasons, planning and the necessary modification 
of these plans is a characteristic common during system planning and 
development. When the system becomes operational, emphasis shifts 
from planning to monitoring. The greater predictability of the outcome 
of decisions and actions permits this shift as well as the more immediate 
need to be concerned with the operation and maintenance of a system in-being . 

The diverse primary objectives and operating characteristics 
have different implications for management characteristics . The 
diversity of these characteristics have implications which affect the 
structure of the management organization, the performance of the re- 
source and information management functions , the nature of the re- 
quirements for management and staff personnel , and the logistical 
support associated with either system development or system opera- 
tion. Figure 27 briefly summarizes some of the implications which 
distinguish the developmental from the operational phases of a system. 
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CHAPTER V 
SYSTEM CYCLE 







V. System Cycle 



A . Introduction 



Thus far, system planning has been treated as if it were a 
static process made up of relatively independent types of activities. 

This is far from the case. In this final chapter of the report, the dynamic 
and interactive character of the system planning and development process 
is emphasized. A framework for organizing and describing both the dynamic 
and interactive characteristics associated with system planning and 
development is shown in Figure 28. The types of information required - 
functional , technical , and administrative are indicated in the three 
rows. The levels of detail, completeness, and finality of the informa- 
tion possible at different system planning and development stages are 
indicated in the five columns. 

Inspection of the figure indicates a twofold orientation to system 
planning. First, the scheme suggests that functional, technical, and 
administrative descriptions are developed throughout all phases with the 
completed descriptions staggered as shown in the figure. This concurrent 
descriptive effort emphasizes the interactive nature of the three types of 
system descriptions. That is, functional or task nature influences the 
technical descriptions and vice versa . The resulting system description 
is a composite of the influences of all three. Second, while the descrip- 
tions are interactive, separate and equal considerations of the functional, 
technical, and administrative aspects of a system provide the basis for 
developing functional or benefit criteria, technical or system effectiveness 

^ The number of phases selected and the particular labels chosen to 
depict these phases was based on prior research experiences with the 
U.S. Navy in the system planning and development area and it conforms, 
for the most part, to their classification scheme. Various authors ( 7 , lo , 38, 
46,82 ) have used different labels and numbers of phases but a careful 
review of their materials suggests that the differences are relatively minor. 
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Figure 28 . The System Cycle 
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criteria , and administrative or management criteria . All three criterion 
sets are needed to fully assess all aspects of computer based information 
systems . 

With respect to the columns of the matrix in Figure 28, the 
column titles help to identify the predominant kinds of activities and 
decisions performed during each phase of the system cycle. System 
formulation pertains to a description of an idea or concept of a system 
which will perform the particular proJect/job. In this phase, the 
description is usually general, incomplete, and tentative in nature. 

The System Definition Phase involves the development of a complete 
but general and tentative description of a system. The description 
includes functional, technical, and administrative types of information. 
The Design and Development Phase represents a translation from primarily 
a paper and pencil exercise to the actual fabrication and installation of 
a prototype model. The Test and Evaluation Phase provides an opportunity 
to check out many of the actual performance/cost characteristics of the 
system against expected values. And, the Operational Conversion Phase 
represents a transition of the system from a prototype status to an opera- 
tional status . A predominant activity during this final phase involves the 
programmed transfer of system management from the system designers to 
system operators . 

With respect to the rows of the matrix in Figure 28, a functional 
description includes information needed about the type of project or job 
the system will perform, the objective of the project/job, the research 
or work phases of the project/job, functions or tasks to be performed 
during each phase, performance standards associated with each function 
or task, and various types of research related constraints which will 
affect the design and/or the operation of the system. A technical 
description includes information needed about the nature of the system 
that will perform the project, which functions will be performed by what 







parts of the system, what hardware/software/personnel configurations 
will perform required functions and specifically how each function will 
be performed, and what cost/effectiveness characteristics are associated 
with the components which make up the configurations . An administrative 
description includes information needed about (1) the organization which 
will manage the system, (2) the environment in which the proJect/job will 
be performed, (3) procedures which will be used to insure financial support 
of the system, and (4) the structural plans which exist for housing or 
containing the system and its various components . 

The material presented in the remainder of this chapter is 
organized into two sections . The next section. Section B, identifies 
the scope and objective of each system cycle phase and summarizes 
major activities associated with each phase. Section C, the final 
section, describes four management aids or tools which are designed 
to facilitate the system planning and development process. 

B. System Phases 



Figure 29 presents a summary of the major activities occurring at 
different phases in the development of an R-P System. Review of the 
figure reveals that the functional description is substantially completed 
by the end of the Design and Development Phase; the technical description 
by the end of the Test and Evaluation Phase; and, the administrative 
description by the end of the Operational Conversion Phase. Using 
Figure 29 as a reference, the following paragraphs define the objective 
and scope of each system phase. 



The objective of the System Formulation Phase is to prepare an 
initial system planning proposal which includes functional, technical, and 
administrative information relevant to the envisioned task-oriented informa- 
tion system. The first column of Figure 29 lists the major activities 
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required to generate the functional, technical, and administrative 
inputs for inclusion in the proposal. The System Formulation Phase 
is initiated at that point in time when explicit and recordable actions 
are first taken by a university scientist (s) or university representative (s) 
to prepare an unsolicited proposal and is completed when the proposal 
is officially submitted to an agency or organization who awards system 
planning and development types of grants . 

The twofold objective of the System Definition Phase is to 
(l) select and delineate an optimum system concept and (2) prepare a 
detailed management plan for conducting and monitoring the remaining 
three System Development Phases . This phase is initiated by the decision 
of the sponsoring agency to award a system planning grant to a particular 
university and is completed when a report is prepared and submitted to 
the sponsoring agency which satisfies the above two objectives. This 
phase is characterized by a large amount of system description and 
analysis activities. Every effort is made to secure sufficient informa- 
tion regarding the functional, technical, and administrative feasibility 
of the proposed task-oriented information system. This phase is most 
critical in that it represents a transition from the less costly paper and 
pencil type of activities to the considerably more expensive activities 
involving the development or acquisition of system hardware, software, 
and personnel components. 

The System Design and Development Phase is initiated when 
a grant is awarded (1) to fabricate and install hardware (or utilize/lease 
locally available hardware) and (2) to design and develop required soft- 
ware. This phase is completed when a prototype model of the system 
exists. Parts of the system's structural and functional properties may 
be simulated in the prototype model rather than involving actual hard- 
ware/software items. This phase may be characterized as being highly 
demanding in creativity, flexibility, energy output, and record keeping. 
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By the end of this phase, the functional description should be essentially 
finalized, the system configuration (s) should be selected and alternative 
feasible components identified. Also there exists during this phase an 
intensive monitoring requirement of the progress being made; and the 
necessary reallocation of resources and changes in the schedule tp meet 
unplanned for events and delays , 

The objective of the Test and Evaluation Phase is to determine 
the engineering and participant feasibility of the prototype system model 
under controlled conditions. In contrast, the objective of the Operational 
Conversion Phase, the last phase prior to operational status of the 
system, is designed to check out the operational (administrative ) 
feasibility of the system. The Test and Evaluation Phase is initiated 
with the design and conduct of engineering tests and feasibility testing 
of the scientist of participant subsystem. This phase is completed 
when the system components and procedures have been firmed up. 

The decision is often made to continue research and development 
activities with some parts of the system. A major decision required 
during this phase is to identify which research project tasks will be 
performed by the system under operational conditions and which project 
tasks will require additional planning and development efforts . 

This final System Development Phase, the Operational Conversion 
Phase, is initiated when the decision is made to convert all or part of 
the prototype system into an operational system and is completed when 
some part of the prototype system achieves operational status . It is 
common with complex information systems to continue planning and 
development activities with those functional or technical aspects of 
the system for which desired types or levels of performance have not 
been achieved. Thus, there usually exists during the Operational Phase 
both planning and development activities as well as system operational 
activities. 
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C . System Planning and Development Aids 

This final section of the report describes four general types 
of aids or tools designed to support the system manager during the 
system planning and development cycle . Although the four management 
aids are of general utility, they are primarily applicable to different 
phases in the system cycle. These four types of management aids 
are discussed next . 

1 . System Evolution 

The phrase "system evolution" as used by various writers 
possesses at least three different meanings. First, the replacement of 
obsolescent components with improved components represents one form 
of evolution; i.e. , substituting improved hardware versions for older 
ones. Second, the application of the modular concept represents a 
second form of evolution. That is, the system is designed such that 
modules (components or subsystems) can be added or replaced as the 
nature of the requirements change or increase. The system is designed 
to be changed and to grow with evolving requirements. Third, within 
the scope of a given system, the various functions performed by the 
system are successively examined to see if the growing technological 
state-of-the-art can be used to achieve a major reallocation of responsi- 
bility from one type of component to another. For example, a system 
planner might allocate to hardware those functions or subfunctions 
which were originally performed by the human; or , allocate to software 
those functions initially performed by a combination of man and hardware . 
This concept of system evolution does not simply involve the replacing of 
a component by the same, but improved version of a component (i.e. , 
hardware for hardware) but the substitution of one type of component for 
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another type of component (i.e. , hardware for man).* It is the third meaning 
that is employed in the following discussion. 

A frustration commonly encountered during the System Formulation 
Phase is the inability to construct a system concept which satisfactorily 
meets the overall desired system objective and scope. The variety and 
number of system related constraints uncovered during this initial 
planning phase are apt to significantly reduce the performance character- 
istics and scope of the desired system concept. Constraints originating 
from functional sources, technological sources, and administrative 
sources may likely force the system planner to select a system concept 
which represents a significant compromise. This type of frustration has 
been a common experience of individuals jassociated with the planning 
and development of complex computer-based infonnation systems. For 
this reason the basic strategy of planned evolution was adopted by 
many information system planners . 

In discussing this problem during one of the site visits, Robert 
Hayes, Director, Institute of Library Research, UCLA, advocated a 
system planning strategy which possesses considerable merit. In 
brief, the procedure involves the development of a system concept 
which completely satisfies the expressed objective and scope of 
the proposed system. Using this as the ultimate goal to attain, the 
identified constraints are arranged in order of their criticality and 
difficulty in overcoming. The range of constraints may include in- 
sufficient information about the functional requirements and character- 
istics of research project tasks and subtasks, limitations in the 
technological state-of-the-art, inadequate facilities to house the system, 
limited money or time constraints , and lack of qualified system planners 
and development management personnel . Given a ranking of constraints , 
the next step is to construct an interim system concept which can be 
readily realized within the known constraints . This provides a continuum 



consisting of the ultimate system concept at one extreme and the 
currently most feasible system concept at the other extreme . Within 
these two extremes , and using the rank ordered constraints , system con- 
cepts and associated strategies designed to successively eliminate the 
various constraints are laid out. That is, each succeeding system 
concept is, in part, designed so as to provide a means or 
strategy for reducing or eliminating some number of constraints . A 
system may go through several complete cycles (planning, development, 
and operation) before the ultimate stated objective is achieved. 

Figure 30 shows a two cycle system evolution. The 

major constraint forcing the two cycle system plan stems from the 

»• 

inability to spell out either the processes or the requirements associated 
with the satisfactory performance of certain research project tasks; 
e.g. , tasks and subtasks requiring a high degree of creative processing. 
As shown in this example, cycle #1 is to be used as a platform for in- 
vestigating these highly internalized research activities with the goal 
of achieving a better understanding and ability to describe these re- 
search related behavioral processes and to uncover suitable standards 
for use in evaluating system outputs. It is quite possible that within 
the time frame of the planned system evolution, some of the constraints 
will not be reduced or eliminated. In which case, the system evolution 
will not - within the planned future - progress beyond some given system 
cycle . 



2 . Flow Charting 

In preparing a system development plan and schedule 
during the System Definition Phase, it is a common practice to build 
the plan around the accomplishment of major system tasks or milestones. 
Resolution of the four classes of decisions discussed in Chapter III form 
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Figure 30. Reallocation of Resear ch Functions 
During the Evolution of a R-P System, 
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the major system planning and development milestones of this proposed 
system planning guide. Figure 31 provides a highly abbreviated 
illustration of major events and decisions involved in selecting an 
optimum system concept. 

The octagonal shaped figures represent major system 
planning decision points, the circles represent events (information 
collection, evaluation of data and information, etc.), and the triangles 
represent starting points, usually possessing stored information or 
prior knowledge on relevant matters . The horizontal lengths of the 
arrows connecting the numbered figures represent the projected time 
needed to accomplish the task/activity from which the arrow originates. 
Direction of the arrows represents output from and input to different 
activities and decisions . More detailed descriptions of various graphic 
techniques useful in system planning are found in the attached bibliography. 
Some relevant references are 38, 55 / 71, 98. 

3 . System Description 

One of the problems which typically becomes apparent 
during the Design and Development Phase concerns the effective use 
of the functional information generated during the earlier system planning 
phases . The form and characteristics of the functional description are 
often not amenable as inputs to the technical design of the system . 

Although the importance of the functional description may be apparent 
to system designers, the problem is how to use the information as inputs 
in designing the technical aspects of the system. For example, while 
there exists a growing body of information and data on the research- 
related behaviors of scientists (26 , 34 , 35 , 54 , 75)/ it is difficult to 
see/ except at a fairly general level of guidance, specifically how the 
documented behavioral characteristics of scientists can be used as inputs 



V-12 



Administrative j Technical Functional 



Figure 31 . Flow Charting Malor Ev ents and Decisions 
in Selecting an Optimum System Concept 







Research functions have already been allocated . 

- Descriptions of research project phases, tasks, subtasks 
are available. 

Information exists on research related attitudes and activities 
of scientists. 

Information exists on university policies , constraints , and intents . 

Information exists on sponsor constraints and guidelines. 

Scientists , technical personnel , and administrative personnel 
contribute to the development of alternative system concepts . 

Identification of sources for criterion data and information . 

Scientists provide project personnel with criterion source 
information . 

Information on responses of scientists to proposed system concepts . 
University administration reviews proposed system concepts . 
Sponsor reviews proposed system concepts . 

Collection and processing of criterion data and information. 
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Figure 31 (Cont.) 



Scientists constraints relevant tq.proposed system concepts . 
University based interface constraints . 

Sponsor provides guidance regarding alternative system concepts . 

Identification of compatible system concepts . 

Development of performance standards for project subtasks . 

Projections made on costs and capabilities of alternative 
system concepts . 

University provides criterion values for cost/benefit measures . 

Sponsor provides guidance in terms of cost/benefit considerations . 

Identifying feasible system alternatives . 

Establishing relative importance to various managerial cost/ 
benefit criteria. 

Development of a rule for combining the weighted managerial 
criteria . 

Selection of an optimum system concept . 
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in designing or fabricating software programs or terminals which inter- 
face with the scientists . 

To illustrate a possible procedure for handling this 
problem. Figure 32 was prepared. The figure is intended to depict 
the relationship of research project (or research-related activities) 
to system-oriented (technical) descriptions as a function of system 
phase. The figure illustrates three aspects of a procedure involved in 
blending functional and technical descriptions. First, the dissimilarity 
of the descriptions is greatest during the initial System Formulation 
Phase, becoming progressively more similar until a single system 
description suffices in the Test and Evaluation Phase. During the 
initial System Formulation Phase, the list of research project tasks 
imply operational or behavioral activities or outputs while the list 
of functions (scientific, management, and informational) imply 
capabilities or processes. The system functions represent broad 
level inferences made about skills and knowledge requirements 
(labelled as functional requirements) covering the range of project 
tasks. Each research task may require more than one class of skills 
and knowledge which are differentially weighted in terms of relative 
importance or demands. At the subtask level, the level of the 
description is such that it is easier to intellectually group the nature 
of the processes or capabilities required to achieve the subtask out- 
put or activities. Likewise, the system labels attached to the sub- 
functions (e.g. , clerical and resource management) are such as to 
imply - at a more operational level - the activities involved. As 
the level of description is more refined to the activity level/event level 
the obvious similarity of the two classes of description is apparent 
since both the functional and the technical aspects are mutually in- 
fluencing the descriptive properties of the other. This brings up the 
second point. The arrows shown in Figure 32 which connect the 
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Fiavtre 32. Relationship of Research Project to 
Technically Oriented Descriptions as a Function of System Phase 
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research project and technical descriptions at each system phase 
indicate that the respective descriptions are mutually influenced 
by consideration of the other type of description. The descriptive 
process is interactive. Thus, at the activity/event level, both 
descriptions are worded in similar terminology, and they merge at the 
Test and Evaluation Phase into a simple system description. Third, 
it is quite difficult to separate the functional from the technical 
(or system-dependent) facets of an operational system. The activities 
associated with the performance of a job or project and the outputs 
are Influenced by the system characteristics . The deceptionally easy 
appearing task of isolating technical and functional system inputs 
has produced its share of problems when working with current opera- 
tional systems . The imposition of administrative procedures on top 
of the Integrated functional and of technical system description further 
complicates the task of deciding - when analyzing an operational system - 
what system inputs, processes, and outputs are attributable to administrative, 
technical , and functional considerations . 

4 . Test and Evaluation Alternatives 

The problem of information system testing and evaluation 
has received a lot of attention and mixed reactions during the past 8-10 
years . A number of excellent articles and books have been written 
on various aspects of information system testing and evaluation . A 
sample of the relevant material reviewed include 9, 11, 14, 16, 38, 41, 46, 
56, 58, 59, 81, 82, 87. Since many of the critical problem issues re- 
garding the evaluation of information systems have already been raised 
and discussed in the preceding chapters, no attempt will be made to 
re-introduce these issues; rather, it would be appropriate in this section 
to point out characteristics common to information system testing and 



evaluation. Below are briefly discussed six of the phenomena found 
to exist with information system testing and evaluation. 

. Testing and evaluation activities are not restricted to the 
so-called Test and Evaluation Phase. System testing and evaluation 
is a continuous process and is performed throughout the life cycle of 
an information system. Because there is a major allocation of resources 
to test and evaluation activities subsequent to the installation of a proto- 
type model, it has become conventional to label this phase as the Test 
and Evaluation Phase. 

. Testing and evaluation may be conducted for one of three 
broad purposes. These broad test objective areas include (1) explora- 
tory and theoretical purposes (e.g. , development and checkout of 
techniques), (2) experimental and applied research purposes (e.g., 
demonstrating the practical application of an information processing 
procedure), and (3) implementation and operational purposes (e.g., 
to demonstrate system performance) . 

. Test and evaluation methods and settings encompass a 
wide range of analytic and empirical techniques . Some of the major 
techniques include mathematical modeling and model exercising, 
functional simulation, non-mathematical analysis (for example, 
personnel judgment), engineering modeling, field testing, controlled 
observation under simulated or sampled operational conditions , use of 
questionnaires and interview techniques, and controlled experiments. 

. Test and evaluation activities span the system hierarchy. 

That is, they include component testing, system configuration testing, 
whole system testing, and multi-system testing. 

. There are three classes of testing and evaluation which 
correspond to the types of system descriptions; i.e. , participant -oriented 
or functional evaluation of the system, engineering -oriented or 



V-18 



f technical evaluation of the system, and management -oriented or 

administrative evaluation. For each class, there are appropriate 
criteria, criterion measures, questions to be answered, and methods 
for obtaining answers to these questions , 

■ , Testing and evaluation activities can be cross -sectional 

in nature or longitudinal. Cross-sectional evaluations are usually 
short in duration and include a representative sample of system 
components or activities . Longitudinal evaluations are designed for 
long duration runs involving continuous or intermittent collections 
of data on the performance of components , configurations , or the 

entire system. 
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APPENDIX A 

RESEARCH PROJECT MODEL 
Introduction 

The first section of this appendix presents two basic variations 
of a research project model: a model based on an experimental or statis- 
tical approach and a model based on a controlled investigation approach. 

The basic research project model is organized in terms of phases, tasks, 
and subtasks . The tasks within each phase represent successive mile- 
stones which must be met to complete the research phase. The subtasks 
are specific activities with definable outputs which are required during 
the conduct of each task. The six general phases include: I. Proposal 
Preparation; II. General Planning (Scheduling Personnel Assignments, 
Facilities); HI. Detailed Planning (Hypothesis Development, Sampling, 
Criterion Development, etc.); IV. Data Collection; V. Analysis and 
Interpretation of Data; and VI. Report Preparation. The experimental/ 
statistical and controlled investigation models vary in their Detailed 
Planning and Data Collection Phase but are identical for Phases I, II, 

V, and VI. 

In the interim report, five factors influencing the type of subtasks 
required during the conduct of Phases III and IV of a research project were 
listed. These included: discipline or subject matter, research objectives, 
research methods or settings, scale of measurement, and types of explanation. 
The first three factors listed were considered to have the most impact in 
shaping the characteristics of the research project. As a preliminary check, 
a representative sample of journal articles from different disciplines were 
reviewed. The second section of this appendix presents a classification 
of the findings from 60 journal articles covering research objective, re- 
search method, and subject area/discipline. 
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Part I. Research Project Model 



Phase I - Proposal Preparation 



Task I-l . Development of Informal Proposal 



Output; Preparation of an informal paper containing 
identification of the problem, preliminary hypotheses, alternative 
approaches, historical information, rationale, and significance of 
the problem. 



Subtasks; 

. Identification of problem (content, methodology, 
technique) . 

. Acquisition, compilation, organization of available 
topic related information . 

. Review, screening, and extraction/tagging of 
relevant chunks of information. 

. Analysis of chunks of information. 

. Formulation of hypotheses . 

. Checking of hypotheses for consistency, meaning- 
fulness and originality against available chunks of 
information . 

. Developing rationale to support hypotheses . 

. Formulating/checking some broad alternative 
approaches/methods for testing hypotheses . 

. Organizing relevant historical background . 

. Drafting of preliminary (pre) proposal . 

. Typing of proposal. 

. Editing of proposal. 

Task 1-2. Determination of Support Sources. Fea sibility, and 
Requirements 



Output; Determination of potential sources of support, 
cost/time/scope constraints , and proposal requirements . 



Subtasks; 



. Identification of potential sponsors - information on 
kinds of research being supported and what work is 
being carried on currently in problem area of interest. 

. Determination of best match between potential sponsors 
and own research interests. 

. Identification of steps and procedures recommended 
by sponsoring agencies for submitting and informally 
discussing proposal ideas. 

. Revision if necessary of pre-proposal based on results 
of above subtasks . 

. Send/carry pre-proposal to sponsoring agency and 
discuss proposal ideas with sponsors . 

. Determine the amount of money that is available 
for research of the proposed type. 

. Identify the sponsors time constraints (six months, 
one year, etc.). 

. Identify formatting and technical proposal require- 
ments and procedures for submission. 

Task 1-3 . Modification/Elaboration/Revision of Pre-Proposal 



Output ; Preparation of a formal proposal based on guide- 
lines obtained from Tasks 1 and 2. 

Subtasks : 

. Spell out, in more detail, the alternative approaches 
identified in Task i-1. Include discussion of pros and 
cons for each alternative approach (rationale) . 

. Search, in more depth, the literature in the selected 
topic area with particular emphasis on finding support 
for or identifying positions which are discrepant with 
the one proposed . 

. Describe in greater detail the hypotheses and their 
underlying rationale. 

. Draft final proposal . 

. Type final proposal . 

. Edit final proposal . 

. Submit final proposal to selected sponsoring agency. 
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Phase II - General Planning 



Task II- 1 . Development of a General P lan/Schedule for 
Conducting the Proposed Effort 



Output; Identification and description of major resources 
and development of a general task schedule for conducting the proposed 
research. 



Subtasks: 

Obtain information on what personnel are available 
for what lengths of time . Include training , experi- 
ence, capabilities, salaries, location (on/off campus) , 

0tC • 

. If consulting help is required, identify and contact 
possible candidates . 

. Determine what facilities are available , where they 
are located, how well they meet the project require- 
ments , how much they cost and for how long they are 

available . 

. Identify information services that are available and 
the lag time and cost characteristics of these services . 
Develop task schedule based on estimates of how long 
and what level of effort is required to complete each 
proposed task. 

. Match personnel capabilities with task requirements 
and assign personnel to perform tasks . 

. Identify tentative end products for each task. 

. Prepare a general plan for making use of available 
facilities and information seiyices . 

. Type task schedule, personnel assignments, and 
plan for use of facilities and services . 

. Submit typed material to university administration, 
support services , colleagues , etc . 



Task II-2 . Modification/Addition/Revision of Pla_n 



Output : Final plan for schedule of tasks, level of effort, 
personnel assigned, and facilities and services to be used. 
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Subtasks; 

, Discuss plan with university administration, support 
services (facilities, etc.), and colleagues to determine 
changes required, etc. 

. Based on feedback from discussions during Subtask 1 
above, modify schedule, facilities to be used, etc. 

. Type up final general plan . 

. Submit general plan to university administration, 
to support services directly involved, and to project 
personnel . 



Phase III ~ Detailed Planning (Experimental and Statistical Variations) 



Task III-l . Identification of Relevant Variables 



Output; Detailed description of the variables that are 
related to the proposed problem area . 

Subtasks ; 

. Review literature for discussions of relevant 
variables and their characteristics - how they 
have been controlled, manipulated, measured, etc. 

. Discussions with colleagues working on similar or 
associated problem areas . 

. Make a list of relevant variables identified from 
literature, from colleagues, from own experience. 

. Determine the underlying characteristics of each 
variable . 

. Determine measurement sensitivity of each variable 
(type of scale - ordinal, interval, etc.). 

. Identify manipulation and control characteristics 
associated with each variable listed (methods and 
results sections of research conducted in the area 
of interest) . 



Task III-2. Development of Detailed Hypotheses 



Output ; Specifically stated hypothesis which identify and 
relate independent and dependent variables . 
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Subtasks; 



. Delineation of subtasks covering Tasks III -2 

through III-8 is a function of the specific research 
objective, discipline, and methodology involved 
in the study. 

Task III-3 . Classification of Relevant Variables,, 

Output ; A list of relevant variables classified with 
respect to their role in the study (independent, dependent, contextual). 

Task III-4 . Selection of a Study Setting 

Output; A description (in detail) of selected study 
setting and the rationale for its selection . 

* 

Task III-5 . Development of Experimental/Statis tical Design 

Output; An experimental design including a description 
of (1) how the independent variables will be manipulated (and compared) 
or their actions recorded and (2) what variables wi )e controlled and 
the method selected for controlling . 

Task III-6. Selection of Sampling Unit 

Output; Selected sample with (1) description of character- 
istics and (2) plan for how subjects or materials are assigned to experi- 
mental conditions. 



The primary distinction between the experimental and statistical 
design is that in the statistical approach, levels of a variable are 
sampled rather than manipulated. 
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TaskIII"7. Criteria Selection/Pevelopment 



Output : A set of criteria and criterion measures to be 
used in the project. 

Task III-8. Design. Development and Checkout of Apparatus, 
and Measurement and Recording Techniques 



Output: Apparatus and initially delineated procedures . 



Task III-9. Development and Pretesting of Data Collection 
Procedures 



Output : Final design of the controlled investigation, 
data collection procedures, and apparatus. 

Subtasks : 

. Set up experimental or controlled investigation design 
and apparatus . 

. Select a few subjects or materials - those that will 
be representative in the experiment or controlled 
investigation. 

. Run each experimental condition or make necessary 
observations . 

. Check for: ease of administration, precision of 
observation and measurement techniques , clarity 
of instructions, etc. 



Phase IV - Data Collection (Experimental and Statistical Variations) 



Task IV"1 . Preparation of Sampling Unit for Study 



Cutput: Subjects/units ready to begin the experiment. 
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Subtasks ; 

. Instructions to subjects, preparation of materials. 

. Do some practice runs , check adequacy of preparation . 

. Allow subjects to adapt to test environment, etc. 

. Specific actions taken depend on - 

- The nature of the sampling unit (living , non-living , 
etc.). 

- The experimental conditions. 

- The nature of the study setting . 

- The method of observation/recording. 



Task IV-2 . Quality Control ^ 

Output: Detection of errors or deviation from plan. j 

I 

Subtasks ; ] 

. Periodically check to see if apparatus is working 
properly . 

. Check on behavior of subjects. 

. See that data are being recorded accurately. 

. Check measurement techniques to see if required 
level of precision is being achieved. 



Task IV-3. Compiling and Summarizing Data 



Output: Data in summary form . 

Subtasks ; 

. Need to know: objectives, hypotheses, experimental 
conditions in order to spell out the subtasks required 
in Task IV-3 . 

Phase III - Detailed Planning (Controlled Investigatipn/Variation) 

Task III-l . Identification of Relevant Variables 

Output : Detailed description of variables that are 
related to the proposed problem area . 
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Subtasks: 



. Review literature for discussions of relevant 
variables and their characteristics (e.g., how 
they have been controlled, manipulated, measured, 
etc.) . 

. Discussions with colleagues working on similar or 
associated problem areas . 

, Make a list of relevant variables identified from 
literature, from colleagues, from own experience, 
previous research, etc. 

. Identify characteristics of the relevant variables - 
range, stability, etc. 



Task III-2 . Development of Detailed Hypotheses 



Output : Specifically stated hypotheses which identify 
(and relate) the variables to be investigated. In a controlled investiga 
tion, the hypotheses is usually derived from one of two major sources: 
(1) a model and (2) previous findings . 



Task III-3 . Classification of Relevant Variables 



Output : A list of relevant variables classified with 
respect to their role in the study (which variables will be controlled, 
which will be measured) . 



Task in-4 . Selection of Study Setting 



Output : A description of selected study setting and 
rationale for its selection. 

Subtasks: 

. List out the relative advantages and disadvantages 
of various environments in which to conduct the 
research project (lab, field, etc.). 
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. Decisions should be made based on: (a) nature of 
behaviors of interest (complex - simple, macro - 
micro, etc.) and (b) study objectives (look at 
one or several variables simultaneously) . 



Task III-5 . Development of Controlled Investigation Design 



Output : A research design including descriptions of 
measurement and control procedures. 

Subtasks : 

. Determination of control procedures . 

. Set up schedule for taking measurements, e.g. , 
time sampling. 

. Set up schedule for systematically varying factors 
of interest. 



Task III-6 . Selection of Sampling Unit 



Output : Description of sampling procedure and character- 
istics of the sample. 

Subtasks : 

. Select samples. 

. Prepare samples for use in the study. 



Task III-7 . Design Development and Checkout of Apparatus 
and Measurement and Recording Techniques 



Output : Apparatus and measurement procedures. 
Subtasks : 

. Consideration of alternative measurement techniques. 
The advantages and disadvantages of the application 
of existing measurement techniques to the factors of 
interest in the study. 
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. Selection/development of measurement instrumentation. 
. Determination of criterion/dependent variables . 



Task III-8. Pretesting of Proposed Instrumentation and Measure - 
ment Techniques 



Output; Final instrumentation and measurement package. 
Subtasks: 

. Run some pretest trials using proposed instrumentation. 
. Pretest measurement techniques on small sample. 



Phase IV - Data Collection (Controlled Investigation Variation) 



Task IV- 1 . Application of Measurement Techniques to Sample 



Output : Measures of factors of interest. 

Subtasks : 

. Prepare sample . 

. Take measurements according to measurement schedule. 
. Record and summarize measurement data. 



Task IV-2 . Quality Control 



Output; Detection of errors and deviation from plan. 
Subtasks: 

. Periodically check on instrumentation to insure its 
proper functioning . 

. Check to insure that data are being recorded accurately. 
. Check measurement techniques to see if required level 
of precision is being achieved. 















Phase V - Analysis and Interpretation of Data 



Task V~1 . Descriptive or Inferrential Analysis of the Data 



Output: Experimental status. 

Subtasks: 

. Subtasks are based on following considerations: study 
objectives, hypotheses, nature of variables , experi- 
mental design, methods, criterion measurement 
sensitivity, etc. 



Task V-2 . Interpretation of Results 



Output : A discussion of how results relate to other 
work in the field and what the implications are - both as contributors 
to the body of scientific knowledge and for practical application (if 
any) . 

Subtasks : 

. Review literature and extract information (results) 
which are similar/different from results obtained 
in present study. 

. Compare results obtained with previous results 
and develop integrated picture . 

. Interpret results with respect to study objectives 
and hypotheses. 

. Determine what practical or potential practical 
application results might have . 



Phase VI - Report Preparation 



Task VI- 1 . Prepare Report Draft 



Output: Report draft. 
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Subtasks: 

. Organize collected material s/inf ormation into 
sections (e.g.. Introduction, Methods, Results, 
Discussion, Recommendations, and Conclusions) . 
. Check formatting requirements (journal, technical 
report, etc.). 

. Type preliminary draft. 

. Review and editing . 

. Type final draft. 

. Quality control the completed draft. 



Task VI-2. Submit Draft Report to Sponsors or Article to 
Toumal Editors 

Output : Modifications and additions suggested by 
sponsors or editors . 

Subtasks : 

. Submit report for review by sponsors or journal 
references . 

. Discuss results with sponsors. 

Task VI-3. Preparation and Submission of Final Report 
(may be in form of journal article, monograph, technical 
report, etc.) 

i 

i 

Output : Final report. 

Subtasks: 

. Modify/rewrite draft report based on recommendations 
from sponsors or journal editors . 

. Final review and editing . 

. Final typing. 

. Final quality control of report . 

. Submit approved report for publication and 
mass dissemination. 
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Part II . Classification of Research Studies 



Three major factors which influence the conduct of a research 
project are (l) the research objectives, (2) the research method, and 
(3) the subject area or discipline. For purposes of classifying the 
research literature reviewed, seven general types of objectives were 
identified . 

. To demonstrate the existence of a phenomenon (e.g. , 
color vision) . 

. To demonstrate the existence of a cause-effect relationship 
between two or more variables . To show that B is a direct 
result of A . 

. To demonstrate a functional dependence between two or 
more variable magnitudes or values . 

. To generate hypothesis concerning the characteristics of 
the phenomenon under investigation . Literature reviews , 
historical studies, and observational studies in "real 
world" contexts may be used to gain insights into the 
nature of a phenomenon . 

. To develop and evaluate a technique or method. The 
technique may be concerned with controlling a variable , 
applying a set of measurements or observing, and recording 
experimental data. 

. To adequately and accurately describe a phenomenon. An 
example of this type of research would be an ethnography. 

. To test the validity and/or reliability of a model which has 
been constructed to represent some process, function, 
or behavior. The accomplishment of this type of objective 
usually involves the application of analytic techniques and 
mathematical formulas. 

The four research method categories include: the experimental 
approach, the statistical approach, the controlled investigation, 
and the analytic approach. The five discipline/content classifi- 
cations selected are Physics and Physical Sciences; Chemistry; 

Biology and Physiology; Experimental and Applied Psychology; 

and Social Psychology, Sociology and Anthropology. These areas were 

used as a basis for sampling the research articles to be reviewed. The 
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number of articles reviewed and categorized per area ranged from 
10 to 15. Sixty articles comprised the entire sample. 

Tables 1 and 2 show the distribution research articles by - 

. Objectives and disciplines. 

. Research method and discipline. 

• Research method and objective. 

In the objective by discipline distribution, it can be seen that 
the studies reviewed in the Physics/Physical Sciences and Chemistry 
content areas tend to follow the same pattern; they are more evenly 
distributed across objective classes than the other three content areas. 
The Biological, Behavioral, and Social Sciences appear to concentrate 
most of their efforts in two objective classes: (1) the demonstration 
of a cause-effect relationship and (2) the demonstration of functional 
relationship. For the research method by content area classification 
the results in Table 1 show that (1) controlled investigation is the 
principle method for Physics/Physical Sciences and Chemistry, (2) 
experimental manipulation is the main method for Biology, Physiology, 
and Experimental and Applied Psychology, and (3) statistical manipu- 
lation and controlled investigation are the methods most used by the 
social scientists Included in the sample . Table 2 shows a high 
relationship (1) between the cause-effect objective class and the 
experimental manipulation method and (2) between the evaluation of a 
technique objective class and the controlled investigation method. 
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Table 1 . Distribution of Articles bv (1) Objectives and Di scipline and 

(2) by Research Method and Discipline * 



Social 

Psychology/ 

Socioloav 




r-^ 




— r 


CO 


r-^ 




r-^ 


UO 


CD 




Experimental/ 

Applied 

Psvcholoav 


r-^ 




uo 


r-^ 


<N 






13 


f-I 


r-^ 




Biology/ 

Phvsioloov 




CD 


1-^ 




<N 








r-^ 






Chemistry 


CO 








<N 


r-^ 


<N I 


1 ^ 




00 




Physics/ 

Physical 

Science 










<N 


r-^ 


CO 


1 ^ 




CD 


CO 


tn 

(U 

u 

X 

0 


Demonstrate a phenomenon 


Cause-effect 


Functional relationships 


Generate hypotheses 


Evaluation of technique/ 
equipment 


Description 


Model test 1 


Research Method 


Experimental 


Statistical 


Controlled investigation 


1 ■ .. ■ ■ ■■■ — — 

j Analytic 



to 

? O >’ 

O* o 
^ (I) 

$ > > 
H 



XI 

o 

•M 

(0 

B 

x: 

o 

XI 

tn 

0) 

o 

•fH 

ti 

(0 

(0 



o 

• (I) 

0) 

3 o 

.S' ® 

O S 

w 

^ .fH 



c 

(0 

5 

0) 

g 



Wi 

0) 

.Q 

g 

3 

C 

(0 



.2, >1 

ja 



0) 

XI 



O 'O 
u C 
0 ) (0 

^ s 
^ !§ 
ca 

0) -ri 
XI ^ 

xj (0 



w 'O . 



>1 

c 

0) 



0) 

(0 

8° 



C 

0) c 
(0 (0 



£ 



.. 0) 
a > 
£ ‘-C 

o 

^ .£, 

8 ^ 
o o 

XI 

0 ^ 
to ^ 
0) >, 

c -Q 

w 

C 

® o 

.u 

1 « 

§.s 



0) c: 
> to 

H 

+j C 

d) i-i 

.Q XI 

o 1o 

(D l-i 

6 csi 
O (o 



Q) 
b n 

^ to 
o 



G 

to 

XI 

0) 

s 

B 

to 

XI 

tn 

0) 



tn 

•fH 

Wi 

O 

O' 

0) 

to 

o 

0) 

c 



r? ^ O. 



O o 

xj CO 

fO T3 



A-16 




iiiiiiiigiiM 






Table 2. Distribution of Articles by Method and Objective 
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APPENDIX B 

INFORMATION -BASED INFORMATION SYSTEMS 
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information-based information systems 

T . Introduction 



Purpose 



ERIC 



As part of the project effort, a detailed description was 
developed of the planning, development, and operation of five 
campus-based information systems . The original purpose for 
obtaining descriptions of these five selected systems was four- 

fold: 

To provide a check of the information system plann ng 

. TO deS.TnVrtn™ti^^^^ hTes ofl rfsS' 

(academic and industrial) throughout the phases of a research 

?rex*^'nd the information system service model . 

Two major problems were encountered in attempting to achieve these 
objectives. The first problem was concerned with obtaining an adequate 
check of the system planning guide. In all five cases, none of the 
system planning processes/decisions had been documented. Additionally, 
it was apparent that the planning and development of these systems was 
primarily guided by constraints rather than by the evaluation of alternative 
courses of action. The second problem was concerned with obtaining 
information about the user population and their requirements . Because of 
the proprietary nature of their research work, the detaUed study of the 
industrial scientists as system users was infeasible. 



* ^ j A fi <?rhumacher, Anne W. An information systejn 

Sinialmittenburg , Vau^an Associates , Incl^niary 1968. 
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System Descriptions 



The five systems and the individuals responsible for their 
planning, development, and operation are presented below. Two of 
these systems, SPIRES and BIS, serve the academic community; two, 

KAS and PENNTAP, serve the industrial community; and one, RICE, 
serves both the academic and the industrial communities . 

, The Regional Information and Communication Exchange 
(RICE) is located at Rice University in Houston, Texas. 
Discussions concerning the planning and developing 
of RICE were held with Mr. Frederick Ruecking. 

. The Knowledge Availability System Center (KAS) is located 
at the University of Pittsburgh. Discussions concerning 
the planning, development, and operation of KAS were 
conducted with Mr. Allen Kent, Mr. Edmond Howie, and 
Mrs. Elizabeth Hartner. 

. PENNTAP, the State Technical Services Center, is located 
at Pennsylvania State University. Discussions were held 
with Mr. Anthony Vennett. 

, The Stanford Physics Information Retrieval System (SPIRES) 
is located at Stanford University, Discussions were 
held with Dr. Edwin Parker and members of his staff. 

. The Brain Information Center (BIS) is located at the University 
of California at Los Angeles. Discussions were held with 
Dr. Robert Hayes and Dr. Peter Amacher . 

A detailed description of each of these systems is presented 
in Part II of this appendix. 



o 
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Classification of Systems 



One way in which information systems can be categorized is 
in terms of their stated objectives. Five general classes of objectives 
have been identified: 



(1) To provide the user with information in the various forms in 
which it is available to the system in response to specific (user 
originated) demands . 
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(2) To provide the user with information in the form desired 
in response to a specific (user originated) demand. In this case 
the form in which the information is presented is not restricted to 
what is avaUable in the system. Thus, the system may create special 
summaries, review articles, etc. taUored to individual requirements. 

(3) To provide the user on a regular schedule with a listing of 
tities and/or abstracts which represent the materials currently acquired 
by the system. One example of this type of system is the Clearinghouse 

for Scientific and Technical Information. 

(4) To provide the user on a regular schedule with a selective 
listing of tities and/or abstracts of the materials currently acquired 
by the system. This selective listing is usually based on an interest 

profile submitted by the user. 

(5) To perform those tasks with, or in support of, the user 
which are aimed at producing a product (e.g. , technical research 
report). These tasks are not limited to the provision of information. 

Of the five systems characterized, two (SPIRES and RICE) 
are representative of the first class of objectives: two (BIS and PENNTAP) 
of the second class of objectives; and one (KAS) of the fourth class of 

objectives. 
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n. System Descriptions 



Regional Information and Communication Exchange 

(RICE) 



Historv/Development of System 
Purpose 



To centralize the bibliographic resources of the Texas 
gulf region and make this resource base available to colleges, universities, 
and industries in the area. 



Background 




- In October of 1964 work was begun to mechanize the 
interlibrary loan system at Rice University. 

- In 1965 the Rice library was offered access to an IBM 
7040 computer (the load on the computerwas small and there was a re- 
quirement to make full use of available machine time) . 

- In 1965 the university received a systems grant from 
the National Science Foundation. This grant is still being continued; 
$13,000 was allocated to the library to study how the library could 
augment the research of the faculty. As part of this study a classification 
of subject materials (places, people, dates, meetings, essays, etc.) 
was developed for a faculty member who was writing a definitive 
bibliography on Thomas Mann. Twenty-six thousand dollars was 
allocated to the library to evaluate the effectiveness of providing 

a SDI service to chemistry faculty using Chemical Abstract tapes. 

- In 1965 information became available on the tele- 
typewriter link that had been set up between Columbia, Harvard, and 
Yale. This kind of system was of great interest. 
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~ In 1966 (February) Project MARC was initiated. Rice 
University was selected as one of the original MARC participants . 

~ In May of 1966 the concept of the Regional Information 
and Communications Exchange (RICE) emerged. This was in part a result 
of an attempt to move the Technical Information Center from the University 
of Houston to Rice University. 

. A grant was received in June 1967 from the 
Department of Commerce to provide information 
to industries in the region. 

. A special purpose grant was received from the 
Office of Education to build up the resource base. 

. A special merit grant was obtained to include part 
of Louisiana in the network. 

. In May 1968 a grant was received from NSF to 
complete and automate the RICE network. 



System Development 



~ The RICE network is to include 18 colleges and universities 
in the Gulf Region. This concept evolved because it was determined from 
an examination of the large university library budgets that no one library 
could afford to acquire all of the required materials. 



~ The colleges and universities to be included in the 

network are: 

. University of Houston 
. Rice University 
. McNeese State College 
. Lamar State College of Technology 
. Del Mar State College of Technology 
. Stephen F . Austin State College 
. Sam Houston State College 
. University of St. Thomas 
. Houston Baptist College 
. Alvin Junior College 
. Victoria Junior College 
. Texas A & I University 
. Loredo Junior College 



o 
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. Texas Southern College 

. Lee College 

. Institute Tecnologico (Mesdco) 

. Wharton Junior College 

- The bibliographic information on the materials held in 
these collections will be included in a central computer file at Rice 
University. 

- Information (resources) in the network will be made 
available to colleges, universities, and industries in the area. 

~ At the present time five colleges are actually 
participating, the remaining 13 are in the process of making formal 
commitments. 

- The only function currently being performed by the 
computer is accounting and billing . 

- Transferring of bibliographic information to tape 
was initiated in May 1968. 

- Requirements for the system include: 

. Fast communication between members of 
network . 

. Identification of what materials are available 
where . 



Marketing 



Major problem is how to inform small industries of the capabilities 
of the RICE System and how to make use of these capabilities . 

Methods of marketing RICE ~ 

- 3,000 newsletters at regular intervals - industries. 

- Newspaper publishers . 

- State Newsletter sponsored by the State Technical 
Services . 
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- Individual seminars. 

- Chamber of Commerce meetings. 

- Thorough SBA. 

- Trips to individual companies (contact is primarily 
with libraries rather than with researchers) . 

Description of Current and Potential User Groups 

Colleges and universities, principally those which are members 
of the network. Ten academic institutions have already made use of the 
service . 

Industries located in the gulf region - mainly large and medium 
sized petroleum-oriented industries . Small industries do not have time 
or money to take advantage of RICE . Forty-seven companies have made 
requests so far. 



Description of Current System 



Personnel and equipment - 

- Personnel 

. Five professionals: director, technical director, 
intern (working on computer applications) , two 
librarians (science information; business informa- 
tion) . 

. Two clerks (help professional librarians) . 

- Equipment 

. 1401 computer. 

. 3 3 SRA teletypewriters . 

. Microfilm readers and printers. 

. Xerox copies. 

Input characteristics - 

- Citations of all material held by the 18 academic 
libraries in the network. 

- LC M^RC tapes. 
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- Industrial citations (when offered by industries in 
the gulf region) . 

Processing of input - 

% 

- Procedure will be to put Rice collection on tape first 
then University of Houston followed by smaller schools. 

~ For each citation the location of the materials will be 
indicated . 

User input - 

- Industry-originated questions: 60% petro -chemical; 
other ones include electronics, math, space, and 
economics . 

- The more sophisticated companies ask the most difficult 
questions . 

- The larger and more sophisticated the company, the 
greater the ratio of copying to searching requests . 

- Approximately 6,000 pages of copy requested since 
December 1968. 

. 800 pages requested by academic institutions 

. 4,500 pages requested by industry 

- Search requests have increased each month since the 
system started. 

- Types of search questions asked of the RICE System - 

. Patent searches (Cameron Iron) . 

. Citation verification (NASA) . 

. Literature search on time sharing of computers . 

. Locate authors . 

. Price of a book subscription. 

. What is a particular journal - refractory journal. 

. Information concerning Texas offshore leasing. 

. Who invented the first mercury manometer? 

. Declorination of fluorine . 

. What is NASA doing in the area of geophysics . 

. Information on inert gas . 

. Information on geothermal steamwells . 

. Information on force winds . 

Processing of user input by the system - 

- If questions are not clear some member of the RICE 
staff calls the user and determines the requirements 
more specifically. 



Matching of queries with resources - 

- At the present time this is done manually by the 
librarian . 

. Once the system is automated all searches 
will be performed by the computer. 

Dissemination of output to users - 

- Mail xerox copies . 

- Teletypewriter to member institutions . 

Feedback from users - 

- Expressions of satisfaction from industries . 

- More requests for copy and searches are received 
each month. 

Charge to users - 

- Advanced membership: $5,000 

. All standard services (access to exchange, 
regional information locating service, inter- 
library loan , etc . ) . 

. $ 1 , 500 worth of photocopying . 

. $1,500 worth of literature search. 

- Special membership: $2,500 

. All standard services. 

. $750 photocopying. 

. $750 literature search. 

- Communicating membership: $1,500 

. All standard services. 

. $450 photocopying 
. $450 literature search 

- Some companies have small deposit accounts . 

- Some companies pay as they use the system. 

- Some companies buy memberships to feed the system. 
(They support it financially but don't make much use 
of it . ) 

- There are nine companies with memberships and six 
companies with deposit accounts . 
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Performance characteristics - 



- Both number of search requests and number of pages 
of copy have increased each month. 



Plans for the Future 



- Put RICE System on self-supporting basis in three years . 

- Put the RICE System on-line eventually. 

- Eventually will charge less to user organizations which 
make contributions to the resource base. 

- RICE should be made one of the distribution points for 
MARC tapes. 

- Within three years most of the data base will be on tape. 

- Eventually integrate Chemical Abstracts SDI Service into 
RICE System. 

- Have 1 8 members of the network buy in different content 
areas . This will maximally increase the regional resource 
base. 

- Integrate Technical Information Exchange (TIE) with 
Regional Information and Communications Exchange (RICE) . 

Organization 



(next two pages) 
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SPIRES 



w^gtorv/DftvelQPmen t of System 
Purpose 

To determine how to meet the real information needs of 
a number of diverse groups of scientist and non-scientist users of 
scientific information. The first group of concern is high energy 
physicists . The ultimate plan is to have Information systems in the 
home which wUl provide content on any subject. The computer is seen 
as a logical extension of the television . 

Development of the System 

- The SPIRES project was undertaken by the Institute 

for Communication Research in 1966. 

- The concept is to develop an on-line interactive system. 

- So far there is no satisfactory operational general 
purpose time-sharing system: SPIRES has designed and is currentiy 

implementing a special purpose time-sharing system. 

- Currently involved in programming bibliographic 

functions . 

- Once bibliographic function is operating satisfactorily, 
start developing other project related functions for the computer (e.g. , 

simulation , data retrieval , etc . ) . 

- Concern with simplification of user-terminal interaction. 

- General concept is that the scientist and the computer 

will work together to make up the system. 

- The current approach is modular development . This 







•• • - ‘ ■ • ■ -., -f i,n-,,^. i, i ^ jj 

approach facilitates modification . The idea is to have a continuously 
evolving system; i.e. » make the system adaptable to the user. 

Description of Users 

One hundred energy physicists working at SLAG 5% of the 
world's high energy physicists. 

Description of Current System 



Personnel and equipment - 

- Staff 

. 1/2 time clerk 

. Three system programmers , one application 
programmer, two part-time student programmers 
. SLAG librarian 1/4 time 
. Ballots senior systems analyst part-time 
. Dr. Parker 
. One secretary 

- Equipment 

. IBM 360-67 
. IBM 360-75 

. IBM 3741 typewriter terminals 
. IBM key punch 



Input characteristics - 

- The current input is composed of the Physics preprints 
obtained and held in the SLAG library collection. (Acquisition rate: average 
50 preprints per week.) 

- SLAG obtains preprints from physicists and from 
Physics laboratories. 

- Eventually users will be able to input their personal 

files . 




Processing of input - 

- SLAG inputs preprint author, title, author's affiliation, 

and other citation information including journal name, volume number, and 

page number for each journal article cited in the footnotes or reference lists . 

* 

Currently this is being done by the teletypewriter terminal . Another 
alternative being evaluated for input is the keypunch. 

- Nuclear Science Abstracts bibliographies, citations, 
and indexing terms . 

User input - 

- The user will query the system for bibliographic informa- 
tion by means of the terminal . 

Processing of user input - 

- Computer processes bibliographic search commands. 

Matching of queries with surrogate files - 

- Computer search programs for matching query terms 
with terms in index file . 

Dissemination of output (search results) to user - 

- Search results are transmitted to user at the terminal 
from the computer. The system is on-line so the user can carry on an 
active dialogue with the computer files . 



* 

This input format permits citation indexing searches forward in time 
using earlier known articles as starting points for the search. 
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Feedback from users - 



“ Behavioral data based on interactions between the 
userand the computer. (Use data will be kept by the system.) 

- User can comment on system performance at the 
terminal by using "show SPIRES" . 

- Questionnaires and interviews. 

Additional sources of guidance for system development and 
modification - 

- Faculty Advisory Committee responsible for advising 

and directing . 

Progress being made on BALLOTS: library automation project. 

- Characteristics of other systems at Stanford which may 
impose constraints. 

- Progress made on similar systems such as MACTIP at 

MIT. 

Organization 

Three projects - 



SLAC <r- ^ SPIRES 




AUTOMATION 

other proje cts at Stanford With Which SPIRES Should IntaoratA 
Project BALLOTS - 

- To process each book, independent of the cost of the 
book. It costs an average of ten dollars. 
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- Supported by Office of Education (OE) . 

- Staff consists of - 

. 4 system analysts 
. 1 programmer 

. 1 librarian 
, some part-time help 

- There are about 40 libraries at Stanford. This includes 
main, branch, and departmental libraries. About 85% of the libraries are 
served by the main library - that is, responsibility for processing. 

- The project has started on the acquisition function - 

should have this completed by January-February next year. 

- They anticipate having the library automated within 

3-5 years - probably 5 years . 

- Much of the thinking on Project Ballots comes from Dr. 

Parker - the on-line interactive philosophy, etc. 

- They acquire about 100,000 documents a year. 

Project information - 

- Sponsored by FORD Foundation. 

- Concerned with automating the administrative and 

accounting functions of the university . 

- There is some pressure to incorporate Project 

Information into Projects SPIRES and BALLOTS. 

Project SLAG - 

- SLAG already had manually processed a pre-print 

collection . 

- Formal arrangements with high energy physicists and 

institutions doing work in this field had already been made. 

- The interviews (depth interviews of 1 hour duration) 

uncovered the interest in having pre-preprints or what is scheduled to 
be done by whom in the way of an experiment . 
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- Another feature uncovered in the interviews is the 

trend towards CRT displays rather than typewriter outputs . 

- Also interviewees wanted personal files and 24 hour 

availability . 

Organization 
(next three pages) 
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Note: Recently projects BALLOTS and SPIRES have been merged 
organizationally. They are Jointly directed by an executive 
committee chaired by the Associate Provost for computing. Other 
committee members include: 

. Director of the Computation Center 
. Director of Libraries 
. Principal Investigator of BALLOTS 
. Principal Investigator of SPIRES 
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Brain Information Service (BIS) 



Historical Information 
Purpose 



To supply information relevant to brain research on 
a national level. 



System Development and Initial Operation 

- BIS was established in 1964 on an entirely manual basis. 

- In 1965 conversion to a computerized system was initiated. 

. Early in 1965 system specifications were 
developed and documented. 

- Problems associated with computerizing system. 

. Lack of qualified personnel - no systems analysts , 
no programmers. 

. Developing a Thesaurus for automatically indexing 
titles . Concept; include every word in title as 
indexable - this created a problem; list of words - 
infinite . 

. Type of computer for which programs were to be 
written was changed three times; 7040; 7094; 
360-75. With each change, new programs had 
to be written for the bibliographic retrieval system . 
The program for the 360-75 should be operational by 
August 1968. 



User Groups 

Approximately 300 users . Half of the users are local (Brain 
Research Institute) the rest are spread around the country (many of these 
were connected with Brain Research Institute (BRI) at one time or another) . 
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Current System Characteristics 



Equipment and personnel 



- 360-75 computer 

- 3 1/2 librarians; area specialists 

- 10 clerical personnel 

Input characteristics (document) - 

- MEDLARS tapes 

- Current journals (punched on cards) 

- Books and journals back to 1960 (punched on cards) 

- All library materials available at UCLA 

- 30 or 40 journals subscribed to by BIS 



User input - 

- Specific questions 

- Requests for bibliographies 

- Current awareness areas of interest 



System processing of user input - 

- Librarians work directly with scientists to determine 
their requirements. 

- Requests for information are currently processed 
manually (i.e. , searches are conducted by librarians 
and bibliographies are compiled). 

- For current awareness, incoming journals are scanned 
by librarian for relevant articles (BIS and UCLA library) . 
For 15 journals the tables of contents are xeroxed and 
sent around. Users can order articles by checking those 
that are desired. 



Additional services offered - 

- Workshops and conferences set up on various topics: 
participants give papers - these are published - editor 
is usually a member of the Brain Research Institute staff. 
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Bibliographies published on specific subject areas. 
Updated reviews in selected areas of neurophysiology. 
This includes all relevant literature (updating is done 
every 4-6 weeks) . 



These reviews are written and updated by members 
of the Brain Research Institute. They may receive 
payment through honorariums , etc . 



Written on three levels of detail; each of which is 
(a) graduate student with general interest - level 1, (b) gradu 
student with specific interest - levels 1 and 2; (c) research 
scientist - levels 1,2, and 3 . 



These reviews will eventually be indexed and 
made available through the computer. 



Provides Index and abstracts for pre-publlshed amdes - 
these are published In Section B of "Communications in 
Behavioral Biology" - 3-6 weeks time la 9 - Section A of 
this Journal Includes articles received within the last 
6-7 weeks. 



Feedback From Users 



Nothing systematic. Based on conversations be^een users 
and librarians. Most users have indicated a high degree of 
satisfaction (particularly pleased with the dissemination o 
xeroxed tables of contents) . 

Two user studies . 



What journals are used. 

What ranks read what (highest readers - associate 
professor level; full professor reads more than 
Instructors and assistant professors) . 



Organization 



Victor Hall 
Director 

-~r~ 

Peter Amacher 
Assistant Director 

1 

Louise Darling 
Bio-Medical Librarian 



Librarians 



1 



Area Specialists 



Clerks 
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Funding 






All money comes from NIMDB. 

BIS 

BIS is part of a network which includes Harvard, Columbia, 
and Johns Hopkins. BIS is concerned with basic content areas; each 
of the other three concentrate on a clinical field . As of now there is 
little coordination. However, plans for the future include increased 
cooperative efforts. 
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Knowledge Availability System (KAS) Center 



History and Development of System 



Purpose 



The Knowledge Availability System Center was established 



in 1963 by the University of Pittsburgh to develop a program of research, 
operations, and teaching. The first major operational program undertaken 
by the KAS Center was fostered by NASA in 1964 as a Regional Dissemination 
Center (RDC) for NASA publications. One original goal of the operational 
program was to make the RDC self-supporting by 1969. The Technology 
Utilization Division of NASA is responsible for the dissemination of 
materials to the regional centers. 

Development and Initial Operation 



and motivated personnel to participate in the development and shaping 
of the system. Once this work was completed, personnel were needed 
who were useful in performing more routine tasks. 



companies in its user group. At one time in the last four years this 
center was serving as many as 75 companies. 

Marketing Procedures 

Steps involved in identifying potential user companies and 
making them aware of the services offered by the knowledge availability 
systems center - 



Initially the system required highly imaginative , creative , 



When the system first became operational it had eleven 



- Select potential user companies from industries 
listed in Dun and Bradstreet . 
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- Make an initial determination concerning the match 
between the system's (center's) current data base 
and the content needs of these industries. 

- Send a brochure (information package) to the selected 

industries. 

- If the response of the contacted industries is favorable 

then a follow-up visit is made . 

- If the visit is successful and. the company(s) wishes 
to subscribe to the services of the center, make up 
a purchase order and have it signed by the company. 

- The signing of the purchase order is followed by a 

visit to the company by a consultant (one of the engineer- 
ing faculty or the Associate Dean of Engineering) who 
works with employees to determine their basic information 



- Sixty-two large and medium-sized industries located in 
Pennsylvania, Ohio, New York, New Jersey, and West Virginia. 

- Colleges and universities in Pennsylvania through the State 

Technical Services Act . \ j * 

- Materials Engineering Magazine (a trade magazine) - used to 

advertise abstracts to subscribers . 

- University of Pittsburgh faculty and graduate students (get 
money from the university budget) . 

Description of Current System 



Personnel and equipment 



- Forty-four staff members 

. Thirteen professionals - content experts, 
systems analysts (seven are full-time) . 

. Eleven consultants - these are all members 
of the engineering faculty . 

. Twenty clerical (sixteen are full-time) . 

- Equipment 

. 7090 (IBM) 

. Xerox 

. ITEK microfiche printer 



needs 



Description of Current and Potent ial User Groups 
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Input characteristics - 

- NASA tapes: STAR and AIAA - include index entries 
and accession numbers (indexing for STAR items is 
done by Documentation Incorporated) • 

- Unit records for all titles in STAR and AIAA. 

- Microfiche of documents listed in STAR (all files 
go back to 1962) . 

- NASA data base - 300,000 documents - 60,000 
are added each year. 

Processing of input - 

- Abstract cards are kept in order by accession number 
back to 1962. 

- Microfiche for STAR documents (N-documents) is 
filed by accession number. No hard copies are 

stored . 

- Four magnetic tapes including index entries and 
accession numbers are used for computer input. 
These tapes are stripped versions of the 18 tapes 
supplied by NASA. There is a substantial amount of 
non -index information on the NASA tapes which is 
not utilized by the Knowledge Availability Systems 

Center. 

- Materials are located at Space Coordination 
Research Center and Hotel Webster Hall. 

User input*- 

- The users supply the center with questions in 
specified content areas. These questions are 
developed by the user and a consultant from 
the system (one of the eleven members of the 
engineering faculty or one of the five subject 
specialists) . 

- A consultant is assigned to each question and gets 
in touch with the user once every three months . 

- At the present time the system is processing 
approximately 700 questions a month. 

Processing of user input by the center - 

- Subject specialists develop search strategies 
from questions using NASA Thesaurus . 



* 





See Figure 1 for user/system interaction . 
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Matching of user-questions with NASA tapes - 



Questions are matched with magnetic tape entries 
using Boolean Search Strategy (selected accession 
numbers are matched with abstract cards by hand) . 
If questions are not receiving any citations from 
NASA a hand search of USGRDR and Nuclear Science 
Abstracts Is performed. 

For most questions an initial retrospective search 
Is performed. For each of the following twelve 
months current awareness searches are performed. 
Reviewers examine abstract cards selected for each 
question and select relevant abstracts . Approxi- 
mately 30% of abstract cards are selected by the 
consultants. 



Dissemination of selected output to users - 



Abstracts are sent one month for the retrospective 
search and each of the following twelve months 
for the current awareness service. 

If documents (hard copy or microfiche) are requested, 
these are provided by the system. For N-documents - 
hard copies are made from microfiche file; for most 
A-documents copies are obtained from local libraries/ 
information centers . 



Feedback from users - 



For each set of abstracts sent users are asked to 
evaluate them for relevance . Currently, approxi- 
mately 80% of the abstracts sent are rated as 
relevant . 

The same forms are used for ordering documents 
from the center (4,114 documents were requested 
and supplied during the first quarter of 1968). 



Charge to users - 



- Type I Service: Subscribers will receive a computer 
print-out which lists the accession numbers of cited 
documents . 



Current awareness - $50.00 per profile. 
Retrospective - $55.00 per profile. 
Current awareness and retrospective - 
$75.00 per profile. 
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- Type II Service: SubFcribers will receive abstracts 
of all documents cited by the computer search. 

. Current awareness - $85.00 per profile. 

. Retrospective -■ $90.00 per profile. 

. Current awareness and retrospective 
$135.00 per profile. 

- Type III Service: Subscriber will receive Engineering 
Review plus relevant abstracts . 

. Current awareness - $180.00 per profile. 

. Retrospective ~ $185.00 per profile. 

. Current awareness and retrospective - 
$275.00 per profile. 

- Type IV Service: Subscriber will receive abstracts of 
cited documents which reflect interests of users in 
general in a subject area. No specific changes can 
be made to satisfy the individual user. 

. Standard interest profile - $100.00 per profile. 

- Profile substitution may be made on a current awareness 
search service for a file of $10.00 per profile. 

- Document charges ~ 

Hard copy - $ .05 per page of original document. 
Hard copy (custom format and size) - $.10 
per page of original document. 

. Microfiche ~ $.50 per document. 

Performance characteristics ~ 

- The center uses the 7090 computer between 2-3 
hours a month. 

- A minimum of six questions per computer run. 

- There were eleven aborts during the first three 
months of 1968. 

. Ten were due to card punching errors . 

. One was due to tape damage. 

- The ITEK reproduces 1 l/2 pages per minute. 

(microfiche to hard copy) 

- Approximately two weeks turn around for initial 
retrospective search. 







Costs 



- Approximately 1/3 million dollars a year is required 
to operate the center. 

- Costs have been broken down into the following 
categories: 

. Professional 
. Clerical 
. Overhead 
. Fringe benefits 
. Travel 
. Computer time 
. Materials 
. G and A 

- Industrial users provide approximately 1/3 of funds . 



Plans for the Future 



Users in addition to current groups - 

- Colleges and universities in Pennsylvania through 
State Technical Services Act (provide abstracts in 
31 areas - 31 standard questions) c (See Table 1 
for standard interest profiles.) 

- Professors and graduate students at the university 
through university budget . 

Expansion of data base beyond NASA - 

- Chemical Abstracts tapes processed by Chemical 
Information Center. 

- Nuclear Science Abstracts tapes . 

- Social Science Data Archives tapes . 

- DDC tapes. 

- Engineering Index tapes . 

Revision of Free Schedule (Table 1) - 

- In addition to charge will be made for documents 
and microfiche requests . 



Table 1 

Available Standard Interest Profiles 



Ceramics, Cermets, Glass 

Elastromers 

Plastics 

Nuclear Materials 

Fiberglass and Fiber-feinforced Plastics 

Composite Materials 

Powdered Metals and Powder Metallurgy 

Lubrication and Lubricants 

Foods 

Forming Techniques 

Machining and Cutting 

Cleaning and Sterilization Techniques 

Joining and Welding 

Adhesives and Adhesion 

Surface Protection and Hardening 

Non-destructive Testing and Quality Control 

Reliability/Life Testing 

Experiment Design (Statistical) 

Instrumentation and Devices for Testing and Measuring 
Analyses for Chemical Composition 
Crystal Growth 

Equilibrium, Constitution, and Phase Diagrams 
Bearings 

Electrical Insulation 
Power Sources 
Bioinstrumentation 
Information Science 
Management Techniques 

Corrosion and Stress Corrosion of Metals and Alloys 
Fluid Flow, Fluid Mechanics, Heat Transfer 









lanlzatlon of Knowledge Availability Systems Cent^ 




- Administrative organization within the university (Figure 2) . 

- Academic organization within the university (Figure 3) . 
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KNOWLEDGE AVAILABILITY SYSTEMS CENTER 
ORGANIZATION — ADMINISTRATIVE 









KNOWLEDGE AVAILABILITY SYSTEMS CENTER 

ORGANIZATION — ACADEMIC 




PENNTAP Library Information System 



Historv/Pevelopment of System 



Purpose 

PENNTAP was established in 1966 to implement some 
of the directives set forth in the State Technical Services Act of 
1965. This act is concerned with supplying technical services to 
industrial organizations . 



Development and Initial Operatio_n 



- The development of the PENNTAP Library Information 
System was carried out by Mr. Anthony Venett. Based 
on his background and prior experience with industry, 

Mr. Venett felt that the most critical function that he 
could perform would be to direct industry to the answers 
to their specific question. 

- Initially three pilot areas in Pennsylvania were selected - 

. York 
. Reading 
. Erie 

- Twenty companies were initially contacted. 

- Industries located in these areas were to contact their 
local commonwealth campus and give them their question (s) 
(there are 20 commonwealth campuses in Pennsylvania). 



Marketing 



Some individual visits have been made to companies. However, 
this is not possible to do on an extensive basis with the currently limited staff. 

- In the summer of 1967 a faculty member at Penn State was 
employed to visit industries in the Erie area and talk to them 
about PENNTAP (approximately 25% return on this). 



Much of the marketing is done at local meetings (e.g. , 
speeches given to Chamber of Commerce, etc.). 

Marketing tools - 

- Nine-minute film on how PENNTAP Library Information 
System helped a Pennsylvania Industry to change 

its procedures and save money. 

- A brochure on what PENNTAP can offer state industries . 

If company is satisfied with PENNTAPS output it wUl continue to 
use the service. 



Description of Current and Potential User Groups 



- Currently there are 95 companies that are using or have used 
this system. Most of these are medium sized companies and 
small research oriented companies . Small manufacturing 
companies are too busy with administrative and production 
related tasks. These small companies have no time for research. 
The large companies have their own library resources . 

- Most of these 95 companies are located in the three pilot areas. 

- No businesses are included. 



Description of Current System 



Personnel and equipment - 

- The staff includes: 1 professional, a secretary, and two 

clerks. In addition, librarians on the commonwealth 
campuses are used, however, they are not paid by PENNTAP 

funds . 

- Xerox machine. 

- Facsimile transmitter. 

- WATS line. 

Input characteristics - 

- Scope: information on technology and applied sciences . 

No patents searching is included. 



- PENNTAP Library has no formal collection of materials . 
Mr. Venett has in his office a few indexing and 
abstracting journals (e.g. , American Society for 
Metals, Metals Abstracts, etc.), also some abstracts 
from Knowledge Availability System Center (31 areas). 

- Principal information resources drawn on by PENNTAP 
include: 

. Carnegie Library: Pittsburgh 
. University of Pittsburgh Library 
. Philadelphia Free Public Library 
. The Penn State University Libraries 
. Hershey Medical Center 

. Federal Libraries (LC , NLM , and Agriculture) 

User input - 

- Questions are submitted to local commonwealth 
campus or to Mr. Venett directly, by telephone, 
or in writing . 

- Content of questions - industrial processes or products. 

- Questions must be specific; general questions are not 
responded to (e.g., state-of-the-art, etc.). , 

- 320 questions have been received over the past 17 
months. Approximately 20 questions are asked each 
month . 

Processing of user input - 

- About 5% of the questions are handled by the 
commonwealth campuses. 

- The remaining 95% of the questions are forwarded 
to Mr. Venett. 

- Generally, Mr. Venett will call the industry to 
clarify the question after he receives it (use 
WATS line) . 

Matching of questions with available information sources - 

- For each question searches are conducted. 

- First step is to consult relevant abstracts and indexes . 

- Searches go back as far as necessary (e.g. , went 
back to 1890's for information on how to build 
windmills) . 

- If no information can be found Penn State faculty 
members may be consulted for possible leads. 



- Once relevant materials are identified, these are 
obtained and scanned. 

- Relevant sections are xeroxed for the user (usually 
does not exceed 20 pages). 

- If government documents are identified the reference 
is usually recommended , however, PENNTAP does 
not buy these documents for its users . 

- Exhaustive background searches are not conducted. 

- If necessary, PENNTAP will provide a list of consultants. 

Dissemination of selected output to users - 

- Xerox copies are mailed to users . 

- For fast service, xerox copies are transmitted over 
facsimile equipment (when only a few pages are needed) 
to appropriate commonwealth campus and commonwealth 
campus contacts the users . 

Feedback from users - 

- A questionnaire concerning relevance of received 
materials was sent out to participating companies. 

- Eighty-five questionnaires were sent out and 69 
returned. No results are currently available. 

- Some critical incidents where industries have 
modified their processing as a result of information 
supplied by PENNTAP. 

Charge to users - 

- All information is provided free to industries. 

Performance characteristics - 

- Of the 320 questions asked during the past 17 months, 

25 could not be answered and four did not deal with 
technology . 

- Average search time to answer a question is five hours . 

- Average turn around time on a question is two weeks . 

- Not enough money or staff to provide current aware- 
ness or state-of-the-art. 
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Costs - 



Budget of approximately $60,000 a year. 









Supplies and publications. 

Wide area telephone (WATS) . 

Postage . 

Instructional supplies. 

Travel . 

Equipment (xerox and facsimile . . . ) 

Salaries 

Wages 

Overhead 



- The university takes 20% of $60,000 for their overhead. 

- Each year submit two budgets: one for continuing at 
same level of effort; one for expanded level of effort. 



Plans for the Future 



- Hire an assistant for next year. 

- Start moving into other areas of the state: Allentown - Bethlehem. 

- Eventual state wide operation . 



Organization 




i 



I PENNTAP ! 
I Anthony Venett 



B-41 







PENNTAP 




* 

PENNTAP will get an assistant technical librarian in the summer, 1968 
with a MS in Library Science. Assistant has had experience in medical/ 
biological areas . 
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Regional Information and Commvinlcatlon Exchange 

(RICE) 
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Knowledge Availability Systems Center (NASA RPC) 
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